Large-scale survey programs usually use complex sampling designs such as unequal probabilities of selection, stratifications, and/or clustering to collect data to save time and money. This leads to the necessity to incorporate sampling weights into multilevel models in order to obtain accurate estimates and valid inferences. However, the weighted multilevel estimators have been lately developed and minimal guidance is left on how to use sampling weights in multilevel models and which estimator is most appropriate.The goal of this study is to examine the performance of multilevel pseudo maximum likelihood (MPML) estimation methods using different scaling techniques under the informative and non-informative condition in the context of a two-stage sampling design with unequal probabilities of selection. Monte Carlo simulation methods are used to evaluate the impact of three factors, including informativeness of the sampling design, intraclass correlation coefficient (ICC), and estimation methods. Simulation results indicate that including sampling weights in the model still produce biased estimates for the school-level variance. In general, the weighted methods outperform the unweighted method in estimating intercept and student-level variance while the unweighted method outperforms the weighted methods for school-level variance estimation in the informative condition. In general, the cluster scaling estimation method is recommended in the informative sampling design. Under the non-informative condition, the unweighted method can be considered a better choice than the weighted methods for all the parameter estimates. Besides, the ICC has obvious effects on school-level variance estimates in the informative condition, but in the noninformative condition, it also affects intercept estimates. An empirical study is included to illustrate the model.
Read
