A CONSTRUCT VALIDATION STUDY OF IMPLICIT AND TIME SENSITIVE VOCABULARY MEASURES

Vocabulary researchers have started expanding their assessment toolbox by incorporating timed tasks and psycholinguistic instruments (e.g., priming tasks) to gain insights into lexical development (e.g., Elgort, 2011; Godfroid, 2020b; Nakata & Elgort, 2020; Vandenberghe et al., 2021). These timed sensitive and implicit word measures differ qualitatively from traditional paper- or accuracy-based vocabulary tests and are believed to tap into lexical strength and representations in the mental lexicon (Elgort, 2018; Godfroid, 2020b). As a result, there have been calls to use both traditional (explicit) and these timed and implicit word measures in a complementary manner (e.g., Godfroid, 2020b; Nakata & Elgort, 2020; Vandenberghe et al., 2021). At the same time, researchers must first develop a thorough understanding of how these different types of measures (explicit vs. implicit and timed vs. untimed) relate to each other before they can make informed decisions on their measurement battery. It is thus well-motivated to examine the construct validity of these measures empirically and systematically. In this validation study, I took the first step to fill this research gap by assessing both the predictive and factorial structure validity of these measures. One hundred and forty-five learners of English took part in five vocabulary tasks: (1) a receptive form-meaning task, where they chose an option representing the meaning of the target word embedded in a sentence, (2) a productive form-meaning task, where they produced the target word to fit a sentence context, (3) a computerized Yes-No (reaction time) test, where they indicated if they knew the target word by pressing keys on their keyboard, (4) a masked repetition priming task with lexical decisions, where they judged if a letter string forms a word in English, and (5) a semantic priming task with lexical decisions. Items in all five tests were the same 40 English words sampled across the 2K - 5K frequency bands. Data analysis involved item inspection and extraction of person-related parameters based on Rasch and/or mixed-effects models. The measures of person ability obtained from individual tasks were then submitted to confirmatory factor analyses in order to assess the psychometric dimensionality of the measure battery. The resulting latent factor(s), representing a pure measure of vocabulary under a specific conceptualization, was then used to predict self-reported proficiency to shed light on their predictive validity. With method effects accounted for, the one-factor solution (“Vocabulary Knowledge”) produces a good fit and is preferred based on the principle of parsimony for both the implicit vs. explicit and timed vs. untimed distinctions. This result provides evidence for psychometric unidimensionality of these measures as representing a potential unitary construct of vocabulary knowledge. At the same time, the vocabulary construct has the most explanatory power (predictive validity) when conceptualized distinctly as lexical knowledge (measured untimed tasks) and strength (measured by timed tasks). Taken together, these results foreground the need for researchers to further specify the nature of the vocabulary construct as well as the operational task features with which it can be assessed empirically. Importantly, I call for more measurement validation work as researchers expand their assessment toolboxes in vocabulary research.