Researchers have investigated the promise of unimodal and bimodal input in enhancing vocabulary learning from meaning-focused activities. Compared to unimodal input, the simultaneous presentation of written and aural input in bimodal input has been argued to direct L2 learners' attention to words and enhance the form-meaning links for new vocabulary (e.g., Long, 2017; Malone, 2018). However, so far, the research focus has been on the learning outcomes, and there is a lack of studies showing the processing of unimodal and bimodal input for vocabulary learning. Moreover, the learning outcomes have traditionally been assessed through paper-based accuracy tests that tap into conscious, verbalizable knowledge of target words. How bimodal input contributes to fluent form and meaning retrieval of the newly learned vocabulary during a real-time activity (e.g., reading) is still not clear and warrants attention. Therefore, the main motivation of this study was to fill these research gaps by comparing the effectiveness of unimodal and bimodal input in facilitating different cognitive processes and developing different types of vocabulary knowledge. Sixty-three adult L2 English speakers were randomly assigned to reading only and reading while listening groups and read a 9500-word graded reader containing 24 pseudowords with or without audio over two days. Cognitive processes were operationalized using two variables: attention (eye fixations on target words) and awareness (reported recall of encountering a target word in the text). Reading processes were recorded using eye-tracking; awareness in learning the pseudowords were probed using retrospective interviews. Learning outcomes were measured with three written untimed tests (form recognition, meaning recall, and meaning recognition) and one sentence-reading test that indicated fluent retrieval of word form and meaning in real time. Four eye-tracking measures, gaze duration, regression path duration, rereading time, and total reading time, were used to assess the effectiveness of reading only and reading while listening on lexical access and context integration of target pseudowords. The results from eye-tracking measures were analyzed using growth curve models; the results from retrospective reports regarding awareness levels were analyzed using mixed logistic models; and vocabulary tests were analyzed using mixed logistic models (form recognition, meaning recall, and meaning recognition) and linear mixed effects models (sentence-reading test). The growth curve models indicated a nonlinear decrease in reading times from the first encounter with target pseudowords to the last encounter. Moreover, the reading behaviors of the two groups differed for the early and late reading processes. The audio in the reading while listening condition augmented the attention to target pseudowords in early reading processes as reflected in gaze durations, whereas reading only condition promoted increased attention to target pseudowords in late reading processes as reflected in rereading times. Both groups were similar in terms of their awareness levels for the target words in the reading text. They also had comparable learning gains on the meaning recall and meaning recognition tests, but the reading while listening group had significantly higher form recognition scores, particularly on the delayed test. Moreover, the reading while listening group had more robust lexical representations for the target pseudowords as reflected in the reading times in the sentence-reading test. These results pointed to a positive effect of access to phonologic forms through audio in vocabulary learning. Overall, the findings of this study provided a broader and more comprehensive understanding of the cognitive processes and the types of knowledge promoted by unimodal and bimodal input.
Read
