This thesis is concerned with the potential applications of cognitive diagnostic models (CDMs) with hierarchical attributes in supporting formative classroom assessments. The conventional CDM approach that requires large sample sizes is impractical in the classroom setting. Three are three CDM-based approaches that do not involve item calibration and thus are practical in the classroom setting: 1) CDM classifications using non-adaptive tests assembled from a calibrated item pool, 2) nonparametric classifications using non-adaptive tests based on CDMs, and 3) computerized adaptive testing (CAT) combined with CDMs (i.e., CD-CAT). Since most CDMs and their applications assume independent attributes, relevant model parameterizations, and the Q-matrix for hierarchical CDMs were discussed. Three studies were conducted to address the test construction and item pool design issues related to the three CDM-based approaches. Specifically, new indices based on the Kullback-Leibler information are proposed for non-adaptive test construction with a calibrated item pool. Different Q-matrix designs were explored for nonparametric classifications, and recommendations regarding the Q-matrix design were provided for teachers. For CD-CAT, an item pool design method based on simulation was proposed and evaluated. The intended contribution of the thesis consists of psychometric tools for the teachers that help them facilitate formative assessments in the classroom and instrumental guidelines for developers of formative assessment systems.
Read
