Face-to-face interaction often involves the simultaneous perception of the speaker's voice and facial cues (e.g., lip movements) making speech perception a multimodal experience (Rosenblum, 2005). Research in second language (L2) speech perception suggests that participants benefit from visual information (Hazan, Sennema, Iba, & Faulkner, 2005; Wang, Behne, & Jiang, 2008) and that perception training can transfer to improvement in production (Iverson, Pinet, & Evans, 2011) and can be generalizable to novel stimuli (Hardison, 2003). Most studies so far have investigated consonants and, despite a couple of studies looking at the multimodal perception of vowels (Hirata & Kelly, 2010; Soto-Faraco et al., 2007), little is known about the effect of multimodal training on the acquisition of vowels. More specifically, no study has looked at the contribution of visual cues in the perception and production of L2 French.The aim of this study is to provide a better understanding of the role of facial cues by examining the effect of training on the perception and production of French nasal vowels by American learners of French. The following research questions guide this study: 1) Does Audio-Visual (AV) perceptual training lead to greater improvement in perception of nasal vowels than Audio-Only (A-only) training does? 2) Does AV perceptual training lead to greater improvement in production of nasal vowels than A-only training does? 3) Does perception accuracy vary in relation to consonantal context? 4) Is training generalizable to novel stimuli?Sixty intermediate American learners of French were randomly assigned to one of the following training groups: AV, A, and control. All participants completed a production pretest and posttest, and a perception pretest, posttest and generalization test. The perception tests (monosyllabic words with various consonantal contexts) were presented within three modalities and with two counterbalanced orders: AV, A, V or A, AV, V. During the three weeks between the pretest and posttests, the AV and A groups received six sessions of perception training.The results of the perception task showed that, contrary to the control group, both the A-only and AV groups improved significantly from the pretest to the posttest, but that the differences between the AV and A-only groups were not statistically significant. When comparing each vowel in each of the three modalities, there was however a trend in favor of the AV training group. The analysis of the consonantal context revealed that, for both training groups, accurate perception of the vowel was higher when the initial consonant was a velar (occlusive non labial) and was lower with palatals (fricative labial). Training was also shown to be generalizable to new stimuli with novel consonantal contexts. In addition, although both groups improved at the production posttest, the oral production of the AV training group improved significantly more than the production of the A-only training group did, suggesting that AV perceptual training (i.e., seeing facial gestures) leads to greater improvement in pronunciation.
Read
