The adoption and implementation of state science standards based on the Next Generation Science Standards (NGSS) have posed significant challenges for the development and interpretation of science Large-Scale Assessments (LSA); specifically, the extent to which the assessment items align with the standards. Previous research has relied on large state or national data sets and classroom assessment data, but has yet to consider students' experiences with the LSA. This research uses qualitative methods to analyze cognitive lab data from Michigan students in Grades 5 and 8 to find evidence of the intended alignment claims of the items designed for the science Michigan Student Test of Educational Progress (M-STEP). The findings indicate that the items elicit the use of the dimensions from students and, in some cases, discriminate among students who chose the keyed response versus those who did not. The evidence of elicitation is often in contrast to the alignment analysis conducted by external reviewers through the Task Annotation Project in Science (TAPS). This work highlights important tensions with which science assessment developers must wrestle and provides recommendations for doing so.
Read
