ABSTRACTMULTIDIMENSIONAL ITEM RESPONSE THEORY: AN INVESTIGATION OF INTERACTION EFFECTS BETWEEN FACTORS ON ITEM PARAMETER RECOVERY USING MARKOV CHAIN MONTE CARLOByJonghwan LeeIt has been more than 50 years since Lord (1952) published "A Theory of Test Scores (Psychometric Monograph No.7)" which is recognized as one of the most influential in Item Response Theory (IRT) history. Since then, there has been extensive research investigating several aspects of IRT such as: (1) Modeling; (2) Estimation of latent traits; and (3) Estimation of item parameters. There has also been extensive development of applications based on IRT such as (1) Equating; (2) Linking; (3) Differential Item Function (DIF); (4) Standard setting; and others. All those applications have the same assumption--that the item parameters are calibrated as accurately as possible. Nevertheless, there has been extensive research investigating the techniques to estimate the item and latent trait parameters. All previously developed estimation techniques are based on the uni-dimensional IRT model. However, estimation procedures have become more sophisticated because of the appearance of multidimensional item response theory models (MIRT). In MIRT, there are several factors that are influential in calibration procedures, such as (1) number of latent traits; (2) correlation between the latent traits; (3) non-normal distribution of latent traits; and (4) different types of configurations of latent traits (approximate simple structure and mixed structure). In this study, the interaction effects of combined factors on item parameter recovery were investigated using the Markov Chain Monte Carlo simulation method. The findings show that a higher number of dimensions require a bigger sample size than lower dimensions--2000 and 1000 sample sizes for 6-dimensions and 3-dimensions, respectively. That model does not consider correlation and skewness in the latent trait distribution, however. This study shows that if there is an additional factor introduced into the features of the latent trait such as correlation or skewness, increasing the sample size is not helpful in improving the accuracy of item parameter recovery. Rather, an alternative MIRT model should be considered in the case of correlated latent traits, and for transforming non-normal distributions of latent traits to normal distributions. a-parameters are more affected when there is correlation between latent traits. d-parameters have more influence when the latent trait distribution is skewed. Overall, the more factors that influence the estimation of the parameters of the MIRT model, the higher the bias found in the item parameter calibration. If the latent structures are independent and normally distributed, then the higher the dimension is in the model specification, the less bias it will have in item parameter calibration. It is also true that if the latent structures have different types configuration, such as AS or MS, then increasing the number of dimensions may possibly decrease the bias created from different types of latent structures configuration. When the latent traits are suspected of having a skewed or non-normal distribution, then it bias is not improved by simply increasing the sample size, though it might be helpful to increase the number of items at the same time. Another way to fix this problem is to use a sample of examinees selected from a wide range of abilities. This is also true in the case of latent traits that are correlated with each other. Selecting the examinee group carefully greatly reduces the bias resulting from the item calibration procedure.
Read
