"Pronunciation presents a significant, persistent challenge to second language (L2) learners (Derwing & Munro, 2015; Piske, McKay, & Flege, 2001). Fortunately, pronunciation instruction works (Lee, Jang, & Plonsky, 2015). However, pronunciation receives relatively little attention in language classrooms. Further complicating the matter are classrooms with learners from diverse linguistic backgrounds and/or differing levels of pronunciation ability, which may impact the effectiveness of one-size-fits-all whole-class pronunciation instruction (e.g., Isbell, Park, & Lee, 2019). Diagnostic language assessment (Alderson, 2005; Alderson, Brunfaut, & Harding, 2014), which prioritizes identifying the specific strengths and weaknesses of learners, is a potentially useful approach to addressing individuals' pronunciation needs. Addressing these issues, I developed a new diagnostic tool for segmental L2 Korean pronunciation called the Korean Pronunciation Diagnostic (KPD). The KPD consists of two sections, production and perception, each with two tasks that tap into phonological knowledge and abilities. KPD feedback includes a list of a learner's most-difficult phonemes to prioritize in instruction and accuracy scores for production and perception of all phonemes. To evaluate the quality and usefulness of the test, I constructed a validity argument (Kane, 2013) for the interpretation and use of KPD scores, which included inferences on the operationalization of relevant theory, evaluation of observations, generalization of scores, explanation of scores with respect to underlying theory, extrapolation of scores to general language use, utilization of feedback by stakeholders, and the usefulness and impact of applying scores. I sought support for these inferences from two main sources: field testing with 198 L2 Korean learners and interviews with 21 learners and one Korean language teacher. Field testing participants completed a background questionnaire, pronunciation self-assessment, independent speaking task, the KPD, and a standardized measure of oral proficiency. Interview participants completed an initial semi-structured interview where they received their KPD score report; 14 learners completed a follow-up interview approximately 3 months later where they discussed recent pronunciation learning activity and took the KPD again. I used several quantitative techniques, including measurement analyses (classical test theory and Rasch), correlations, cluster analysis, to analyze the field testing data. I analyzed interview data qualitatively. Support for the operationalization, generalization, and explanation inferences was strong, supporting the interpretation of KPD scores as strengths and weaknesses in the production and perception of Korean phonemes. Support for the extrapolation inference was positive but limited. Correlations between KPD scores and learner self-assessments were positive but not large, as learners had limited awareness of their fine-grained pronunciation abilities. Similarly, the KPD's discrete and delimited measurements of phoneme accuracy had limited overlap with pronunciation in spontaneous, meaning-focused speech. The utilization inference was well-supported, though improvements to the KPD score report could further enhance stakeholder interpretation of results. Finally, positive but limited evidence for the usefulness and impact of the KPD was found: Findings suggest that learner application of KPD results has the potential to support pronunciation development, but this is conditional on learner effort. I determined that more evidence is needed to sufficiently support this inference. Overall, the interpretation and use of KPD scores is supported, but future development and research efforts should focus on the effective application of the KPD's diagnostic feedback."--Pages ii-iii.
Read
