Language in multimodal writing processes and performance : developing multimodal writing tasks for L2 learners

In this sequential mixed methods research project, I first investigated learners' needs of multimodal writing in the higher education setting and then examined adult L2 writers' multimodal writing performances and processes, and their task perceptions. For the first study, a qualitative needs analysis, I conducted individual semi-structured interviews with 7 instructors of undergraduate courses in different disciplines to explore how they implement and perceive multimodal course assignments. Additionally, I collected 161 course syllabi data from which I identified 104 multimodal tasks. Triangulating two data sources, I found three themes that emerged from the two data sources: (1) goals and instruction of multimodal writing: disciplinary versus creative expression; (2) linguistic mode in multimodal texts: written and spoken words; and (3) tasks of multimodal writing: individual versus collaborative work. Based on the needs analysis, I developed a timed multimodal writing task that is to construct a narrated slide presentation. I utilized this task as one of the instruments in the subsequent phase.In the second study, adopting a convergent parallel mixed methods design, I investigated L2 learners' multimodal writing performances and processes and their perceptions toward the multimodal writing task. Thirty-one adult Korean learners of English intermediate- to high- proficiency individually completed a multimodal writing task (i.e., a timed argumentative narrated presentation task) and a monomodal writing task (i.e., a timed argumentative essay task) while their on-screen writing behaviors were screen recorded. After the multimodal writing task, each writer completed a stimulated recall interview in their first language (Korean) watching a video of one's own writing processes. They all completed task perception and background questionnaires. Writing process data-on-screen writing processes and stimulated recalls- were qualitatively analyzed using an inductive approach. In terms of task performances, five experienced academic English instructors evaluated the multimodal writing performances in terms of the overall quality, visualization quality, and language; and three of them also rated the monomodal performances using an analytic rubric.Findings from the performance data revealed that multimodal text quality is strongly associated with language performance, but another dimension of nonlinguistic performance also contributes to the overall text quality. More specifically, the multimodal performance data fit a regression model that explains 83% of the variance of multimodal text quality with language scores (s = .62) and visualization scores (s = .45). Furthermore, the language scores of participants' multimodal writing performance showed significant positive correlations to all subscores and total score of monomodal writing performances with moderate to strong effect sizes; however, none of the scores of monomodal writing task performances were correlated with the visualization score of the multimodal writing task performances. From the writing process data, I found that L2 writers spent a smaller amount of time and effort on constructing visual texts than language, especially in the middle of the text construction processes. When focusing on language, they spent considerable time on selecting and upgrading words for scripts and evaluating information they found from the Internet and their own texts-constructed-so-far. Results from the task perception data showed that the multimodal task was was perceived to be more complex, difficult and interesting than the monomodal task. I discuss implications for L2 writing research and pedagogy focusing on how to understand multimodal tasks as language tasks for learners whose goal is to improve language.