EVIDENTIARY VALIDITY OF THE EDUCATION QUALITY AND ACCOUNTABILITY OFFICE’S MATHEMATICS ASSESSMENT

EVIDENTIARY VALIDITY OF THE EDUCATION QUALITY AND ACCOUNTABILITY OFFICE’S MATHEMATICS ASSESSMENTBy Nazlı Uygun Validity of a measurement refers to appropriate test score meanings, uses, and interpretations (Messick, 1989; Kane, 1992). There are different approaches to validity: an evidentiary aspect of validity is one requiring gathering statistical evidence to evaluate test score meaning. A common approach to validation is comparisons of test score equity across different population groups. This research examines the evidentiary validity of mathematics test administrations by the Education Quality and Accountability Office in Ontario, Canada. Using factorial invariance and differential item functioning analyses, score validity was investigated across both achievement level and gender groups. Validity evidence was provided via a four-step measurement invariance procedure and differential item functioning analyses. However, items causing invariance problems and/or functioning in favor or against a certain population group are identified through the analyses. These violations are not necessarily a threat to construct validity but provide guidance for revisiting the test framework and revising some of the items. Content strands functioning differently for gender groups are mostly consistent with conclusions in the existing literature. Practical suggestions for measurement equity are discussed after reporting statistical findings. To reduce achievement gaps between student groups, future studies are needed to identify items causing invariance obstacles, presenting partial invariance solutions, and revising items with differential item functioning.