This dissertation is a collection of three papers that each use Student Response Theory (SRT), a method for estimating educator effectiveness built upon an analogy of students and teachers to test items and examinees in Item Response Theory (IRT). Prior studies compare SRT to competing methods like value-added and student growth percentile models. This study broadens the literature by shifting the primary focus to comparisons between SRT and IRT and implications of their differences in the context of teacher evaluation and accountability. Using student-teacher linked data from an anonymous large school district in a major U.S. city, unique contributions of IRT concepts and procedures are applied to SRT. The first paper focuses on the construction of a student instructional demand index that operates like an item difficulty parameter in an SRT model. Although prior studies recommend two different methods for estimating this index, they focus primarily on one method (regression analysis) and minimally on the other (IRT calibration). In this paper, I select indicators of instructional demand that are optimal for IRT calibration and compare instructional demand indices across model specifications and estimation methods. I find that the IRT calibration method yields estimates of instructional demand that are more consistent with other indicators of teacher quality than the regression analysis method. However, when students and teachers are evaluated against a difficult standard, neither type of instructional demand index is consistent with these other measures.In the second paper, I define mathematical functions of SRT model parameters that are comparable to characteristic curves and information functions in IRT. From these functions, I derive several different indicators of teacher performance and compare their respective meanings and the quality of information they each provide. I then identify groups of teachers whose classes comprise equivalent levels of instructional demand, conceptually similar to parallel test forms in IRT. I find high levels of consistency between different effectiveness measures from the same SRT model. The results also suggest that state-determined thresholds for proficiency and advanced proficiency are so difficult for most students in this district that they provide little information about educators. The basic proficiency threshold, however, is informative about most students and teachers.The third paper applies procedures related to differential item functioning, common item equating, and item selection to the SRT framework as potential tools in a formative teacher evaluation system. Tests of "differential student functioning" find consistent model performance across most subgroups of teachers and classrooms. Although SRT models are successfully equated to a common scale across years, some estimates of teacher growth have unreasonably large standard errors, offering little power to make inferences about changes in performance. Mismatch between the abilities of teachers and demands of their students are the main factor driving this finding. Applications of optimal item selection procedures to SRT, however, may be useful for matching students and teachers with one another in mutually beneficial ways.These findings are generally encouraging, but also emphasize the importance of setting reasonable goals for students and teachers and thoughtful assignment practices in determining whether this method will be useful in practice.
Read
