I have participated in a guest lecture by Professor Norbert Schmitt from the University of Nottingham (UK), that I found interesting. The lecture was held at the University of Oslo in conjunction with a network meeting of LUNAS (Language Use in Nordic Academic Settings) and was supported by the Nordplus Nordic Languages Programme and the Department of Linguistics and Scandinavian Studies.
Schmitt focused the lecture around four main questions related to assessment of vocabulary:
1) Why do you want to assess?
2) What words do you want to assess?
3) What kind of knowledge do you want to assess?
4) Which item format do you want to assess?
The lecture was given in a very interactive manner and the participants actively participated to contextualize the talk from different perspectives. Below I will give a very short summary of what I have taken away with me from participating in that lecture.
Under the heading Why do you want to assess/test?, Schmitt mentioned a number of different purposes of assessments. The purpose of the measuring could e.g. be to look at achievement, for motivation, to see what the students already know, to identify shortfalls in lexical knowledge, diagnostic, proficiency, to look at the short term or long term effect of an intervention etc. This means that the purpose therefore differs according to context but whether it is related to practice or research the purpose may not only be related to vocabulary knowledge but also the language use. Therefore not only a vocabulary score should be considered but rather what the vocabulary score means.
Determining What words do you want to assess depends on the purpose of the assessment but Schmitt also clearly highlighted that word frequency may be the best tool to decide on which words to assess (e.g. the COCA corpus; http://corpus.byu.edu/coca/). He realized this is not a perfect tool for selecting relevant words to include but suggested it to be the most relevant. For younger children, however, a written corpus will make no sense and the corpus should rather be based on talks occurring between teacher/pre-school teacher and their students/children, and talks between parents and their children, as these may then be preferable.
What kind of knowledge should be assessed depends on the purpose of the assessment. Schmitt referred to Nation ‘s word knowledge aspects (2001) and the component form, meaning and use which originally may have been adapted from Bloom and Lahey (1978).
Which item format to assess is also related to the purpose of the assessment. It could be e.g. multiple choice item format, matching vocabulary, recall item format by context or by definition. Schmitt’s examples were taken from work with second language learners and students, no examples from young children were given. However, a work in progress by Benjamin Kremmel looking at the relationship between clinical test results and the results from interview/talk with the informants may possibly give the same result across age?
Kremmel found ca 20% overestimation of the vocabulary scores when multiple choices and matching were compared to the interview. While recall definition and recall context result in an underestimation of scores by approximately 15-19%. Low frequency words result in guessing and both overestimating and underestimating occurs. This means that considering how to report results from vocabulary assessments is very important. Instead of reporting that an individual knows words or has learnt words (which may include derivations and collocation), and assuming that the words are known and used regardless of contexts, we rather should refer to the specific context in our reporting of the results.
Finally, Schmitt also referred to an article he had written together with Zimmermann (2002) focusing on word families versus lemmas. In their study they found that we cannot make the assumption that word families are known by students (e.g. noun, verb, adjective and adverb/persistence, persist, persistent, persistently). The students had higher scores on recognition compared to expression of items but different word forms were directly accessible for the students in general. Therefore Schmitt suggested lemmas to be an actual compromise between word families and individual words.
I think Schmitt gave a good and very engaging talk about a very interesting topic. However, I found that a very important question was missing in his talk, namely - who would you like to assess? Typically there is little focus on the group under investigation but to reveal assessment results of high quality the group under investigation should also be considered.
Bloom, L. & Lahey, M. (1978). Language development and language disorders. New York:
John Wiley & Sons.
Nation, I.S. P. (2001). Learning vocabulary in another language. Cambridge UK: CambridgeUniversity Press.
Schmitt, N. & Zimmermann, C. B. (2002). Derivative Word Forms: What Do Learners Know?TESOL Quarterly, 36(2), 145–171. doi: 10.2307/3588328
-Kari-Anne B. Næss-