In common educational research settings, a latent achievement construct for the student is measured at both the beginning and the end of a learning program so that the added value of the instructional program can be quantified. Explanatory variables at both student level and school level are used to help explain the change in levels of performance on the construct. Test scores, which serve as a proxy for students' true achievement, contain estimation error. Some of the explanatory variables may also be latent variables and they are subject to error as well. Traditionally, the item response theory (IRT) estimates of the latent variables are obtained before they are entered into linear regression models. The problem with this two-step approach is that the relationship between dependent and independent variables could be distorted due to error in IRT score estimates. To address this problem, this dissertation proposes a combined IRT and multilevel model which estimates achievement and the added value simultaneously. Bayesian MCMC (Monte Carlo Markov Chain) method is used to fit the combined model. The performance of the combined model and the one-step Bayesian approach are compared with the traditional two-step approach on simulated data. The simulations are carried out on a simple linear model and a multilevel model as well. The results indicate that this one-step approach recovers the true relationships among latent variables with less bias and more accuracy. Following the simulation, the study applies the new approach to an empirical dataset obtained from the Mathematical Education for Elementary Teachers (ME.ET) project. Special attention is given to the missing data issue in the application. The final chapter of the dissertation discusses the sources of IRT score error and its implications.
Read
