In this dissertation, I documented the effectiveness of a particular test-taking condition in the current task-based performance testing (e.g., TOEFL iBT, IELTS, OPI); namely, planning time (the time given for test takers to plan their responses before actually performing). In assessment contexts, the construct of planning addresses both language and test-related theories; in terms of the latter, it is associated with components constituting test qualities (test validity, authenticity, and fairness; Wigglesworth & Elder, 2010). This is because planning time is already a critical test accommodation, or a task implementation condition, as termed from the task-based research paradigm. Indeed, researchers and test developers suggest planning time as a major component in determining task difficulty in that the varying planning conditions are appropriate to represent the items’ different cognitive-demand levels (Norris, 2009; Robinson, 2001). Therefore, I explored whether test takers’ efficient use of varying planning times is contingent upon test-task characteristics in the context of the TOEFL iBT Speaking test, in which which varying degrees of task conditions such as planning, and test-task types co-exist.Ninety-nine Korean university students took three speaking tests, which each consisted of one independent task (impromptu task) and two integrated tasks (reading-listening and listening-only tasks). As in operational testing, independent tasks were given 15 seconds to plan while the two integrated tasks were given 30 and 20 seconds, respectively. For each test set, participants performed under a specific planning condition: namely, Unguided planning (planning without specific instructions), Guided planning with Writing (planning with instructions given as to write to plan), and Guided planning with silently thinking (planning with instructions given as to think or outline silently to plan). After each test, test takers partook in a series of surveys and interviews to reflect on the appropriateness of each task’s planning times. Subsequently, I undertook multiple methods (quantitative and qualitative) on the collected data points. Three independent raters scored a total of 891 speech samples according to the TOEFL iBT speaking rubric. Two trained coders coded the speech samples for the three discourse quality measures pertaining to complexity, accuracy, and fluency. I thematically analyzed survey and interview responses through NVivo. Findings indicated that participants’ performance and perceptions were directed by the influence of test-task characteristics regardless of the planning activities they made use of. Their test performance and speech quality varied from independent tasks to integrated tasks; they generally scored lower, and demonstrated slower speech rate, increased lexical errors, and simplified language in independent tasks. In addition, test takers believed that extended planning time was unnecessary for integrated tasks, for the reading and listening sources were readily applicable to actual responses; yet 15 seconds did not suffice for them for the independent tasks to familiarize themselves with the given prompt. I discuss study results and conclude the dissertation by making connections between speech-production planning theories in task-based research (Robinson, 2001; Skehan, 1998) and planning time practices in second language assessment (Elder & Iwashita, 2005; Wigglesworth & Elder, 2010).
Read
