NONPARAMETRIC MULTILEVEL LATENT CLASS ANALYSIS WITH COVARIATES: AN APPROACH TO CLASSIFICATION IN MULTILEVEL CONTEXTS   By Chi Chang     
  A DISSERTATION Submitted to Michigan State University in partial fulfillment of the requirements for the degree of  Measurement and Quantitative Methods ΠDoctor of Philosophy  2016     ABSTRACT  NONPARAMETRIC MULTILEVEL LATENT CLASS ANALYSIS WITH COVARIATES: AN APPROACH TO CLASSIFICATION IN MULTILEVEL CONTEXTS   By  Chi Chang  The dissertation  investigated how classification accuracy and parameter estimation of a nonparametric multilevel latent class analysis with covariates (hereafter referred as to conditional NP-MLCA) are affected by six study factors: 1) the quality of latent class indicators (i.e., conditional response probabilities: CRPs), 2) the number of latent class indicators, 3) level-1 covariate effects, 4) level-2 covariate effects, 5) the number of level-2 units, and 6) the group size of level-2 units. A total of 384 conditions were examined. Among the conditions and study factors explored in this dissertation, the results suggested four important implications. First, level-1 classification accuracy was acceptable when the sample size is 9,000, quality of indicators is 0.8, and the number of latent class indicators was 12. Second, the covariate estimates can be extremely biased when the quality of indicators was 0.6, especially when only six indicators were used. Third, for the level-2 covariate effect, larger samples size can compensate for the condition when the CRPs of indicators were 0.7, given the conditions explored in this dissertation. In addition, among the conditions where CRPs of indicators was 0.8, the biases of level-2 covariate effects were smaller when the number of groups was 150 than those when the number of groups was 50. Fourth, when the latent classes had indicators that showed consistent patterns, CRPs (0.6, 0.7, and 0.8) had the strong effect on CRP estimates in terms of biases and 95% CI coverage rate. When the latent classes had indicators that showed mixed patterns, the number of indicators (6 vs. 12) had the strongest effect in terms of biases and   95% CI coverage rates of CPR estimates of latent classes. An empirical study was included to illustrate the model.            
       Copyright by CHI CHANG 2016 iv          
  
 This dissertation is dedicated to Mom and Dad. Thank you for always believing in me, being proud of me, and loving me unconditionally.  
   v  ACKNOWLEDGMENTS    There are many people who have given me much more than I repay.  My deep gratitude goes first to my advisor, Dr. Kimberly S. Maier who has given me cheerful encouragement, tremendous support, and superb mentorship throughout my doctoral study. She always understands me, opens to my inquiry and continuously encouraged me to explore my area of interest. This dissertation would not have been completed without her fruitful advice and generous support. Her guidance and constructive feedback enriches my knowledge in methodology and inspired me to be a rigorous scholar. I am privileged to be her advisee and truly indebted to her.  This dissertation also owes its existence to thoughtful prodding and wise counsel of Dr. Richard T. Houang. For the last seven years, he has become my mentor. I am grateful to him and deeply appreciate for his rich guidance, enormous support, fruitful comments, and great patience. His methodological knowledge, rigorous attitude, and insightful thoughts on research have profound influence on me. I will miss each conversation I had with him.  My special thanks go to Dr. Joseph Gardiner for his invaluable comments and great support to my dissertation. I am grateful to him for many things he has done for me. It was Dr. Gardiner who built a bridge connecting me to my dream.  With all my heart, I would like to thank Dr. Tenko Raykov who, throughout the doctoral studies, has given me his considerable encouragement and has unwavering believes in me. Working with him has been a very inspirational experience. I have gained invaluable feedback, vi  exceptional insights, and incredible support from him. Without his help and support throughout this journey, I couldn™t have been achieved what I have today.  My sincere gratitude goes to Dr. Laurie Van Egeren and Dr. Jamie Wu, for their help on accessing the dataset. I have enjoyed and inspired by the discussion with them and the experience helped shape my perspectives on educational issues in real world.  I would like to express my gratitude to my editor, Daniel MacCannell, for his editing assistance and my colleague, Xiaoran Tong, who provides technical help to this dissertation. I would also like to thank my cohorts, and friends in the CSTAT, the MQM, and the Biostats program, in the United States, Taiwan, Korea, Burkina Faso, and China, who have helped me in countless ways throughout my years at the Michigan State University.  Last but not least, my deep and heartfelt gratitude goes to my lovely family, my mom, Chien Chen, my dad, Tai-Nian Chang, and my sister, Li Chang. Thank you for supporting my every decision without questions, and not complaining about my absence in your life throughout these years.      vii  TABLE OF CONTENTS   LIST OF TABLES ......................................................................................................................... ix  LIST OF FIGURES ...................................................................................................................... xii  Chapter 1   Introduction .................................................................................................................. 1 1.1 Research Goal ....................................................................................................................... 4  Chapter 2   Latent Class Analysis and Multilevel Latent Class Analysis ....................................... 6 2.1 Latent Class Analysis ............................................................................................................ 6 2.1.1 Model Specification ........................................................................................................ 6 2.1.1.1 Local Independence ................................................................................................. 7 2.1.1.2 Latent Class Probability ........................................................................................... 8 2.1.1.3 Conditional Response Probability (CRP) ................................................................ 8 2.2 Nonparametric Multilevel Latent Class Analysis ................................................................. 9 2.2.1 Model Specification ...................................................................................................... 10 2.2.2 Conditional Independence Assumption ........................................................................ 13 2.3 Adding Covariates in Nonparametric Multilevel Latent Class Analysis ............................ 14  Chapter 3   Model Estimation: The Use of the Expectation-Maximum (EM) Algorithm in NP-MLCA ........................................................................................................................................... 17 3.1 The Likelihood Function ..................................................................................................... 17 3.2 EM Algorithm for Parameter Estimation ............................................................................ 19  
Chapter 4   Simulation Design and Outcome Measures ............................................................... 22 4.1 Simulation Design ............................................................................................................... 22 4.1.1 Data Generation ............................................................................................................ 22 4.2 Evaluation Criteria .............................................................................................................. 24 4.2.1 Classification Accuracy ................................................................................................ 25 4.2.2 Bias ............................................................................................................................... 25 4.2.3 Standard Error Bias ...................................................................................................... 25 4.2.4 Coverage Rate ............................................................................................................... 26 4.3 Six-way ANOVA ................................................................................................................ 26  
Chapter 5   Simulation Results...................................................................................................... 28 5.1 Summary of the Results ...................................................................................................... 31 5.2 Classification Accuracy....................................................................................................... 34 5.2.1 Individual-Level Classification Accuracy .................................................................... 34 5.2.2 Group-Level Classification Accuracy .......................................................................... 37 5.3 Parameter Estimation Recovery .......................................................................................... 40 5.3.1 Bias ............................................................................................................................... 40 5.3.1.1 Level-1 Covariate Effect on Latent Class 1 ........................................................... 40 viii  5.3.1.2 Level-1 Covariate Effect on Latent Class 2 ........................................................... 43 5.3.1.3 Level-2 Covariate Effect ........................................................................................ 45 5.3.1.4 Conditional Response Probability of Latent Class 1 ............................................. 48 5.3.1.5 Conditional Response Probability of Latent Class 2 ............................................. 51 5.3.1.6 Conditional Response Probability of Latent Class 3 ............................................. 54 5.3.2 The Standard Error Estimate Bias ................................................................................ 57 5.3.2.1 Level-1 Covariate Effect on the Latent Class 1 ..................................................... 57 5.3.2.2 Level-1 Covariate Effects on the Latent Class 2 ................................................... 60 5.3.2.3 Level-2 Covariate Effect ........................................................................................ 64 5.3.2.4 Conditional Response Probability of Latent Class 1 ............................................. 67 5.3.2.5 Conditional Response Probability of Latent Class 2 ............................................. 70 5.3.2.6 Conditional Response Probability of Latent Class 3 ............................................. 74 5.3.3 95% Confidence Interval Coverage Rate ..................................................................... 77 5.3.3.1 Level-1 Covariate Effect on the Latent Class 1 ..................................................... 77 5.3.3.2 Level-1 Covariate Effect on the Latent Class 2 ..................................................... 80 5.3.3.3 Level-2 Covariate Effect ........................................................................................ 84 5.3.3.4 Conditional Response Probability of Latent Class 1 ............................................. 87 5.3.3.5 Conditional Response Probability of Latent Class 2 ............................................. 91 5.3.3.6 Conditional Response Probability of Latent Class 3 ............................................. 95  Chapter 6   Empirical Study ........................................................................................................ 100 6.1 Data Collection, Measures, and Covariates ...................................................................... 100 6.2 Results of the Study........................................................................................................... 104  
Chapter 7   Summary and Discussion ......................................................................................... 109 7.1 Summary and Implication of the Simulation Finding ....................................................... 109 7.2 Summary of the Empirical Study ...................................................................................... 114 7.3 Discussion ......................................................................................................................... 115 7.4 Limitation .......................................................................................................................... 118 7.5 Future Studies .................................................................................................................... 122  
APPENDICES ............................................................................................................................ 125 Appendix A Template Mplus Syntax ....................................................................................... 126 Appendix B Descriptive Tables .............................................................................................. 128  
REFERENCES ........................................................................................................................... 168    ix  LIST OF TABLES   Table 1. Summary of Important Effect Sizes from Six-way ANOVA Results ................................ 33 Table 2. Study Factors with Strong Effect Sizes on Level-1 Classification Accuracy .................. 36 Table 3. Study Factors with Strong Effect Sizes on Level-2 Classification Accuracy .................. 39 Table 4. Study Factors with Medium Effect Sizes on Biases of Level-2 Covariate Effects .......... 48 Table 5. Study Factors with Strong Effect Sizes on Biases of Conditional Response Probabilities of Latent Class 1 ........................................................................................................................... 51 Table 6. Study Factors with Strong Effect Sizes on Biases of Conditional Response Probabilities 
of Latent Class 2 ........................................................................................................................... 54 Table 7. Study Factors with Strong Effect Sizes on Biases of Conditional Response Probabilities 
of Latent Class 3 ........................................................................................................................... 57 Table 8. Study Factors with Medium Effect Sizes on Standard Error Biases of Level-1 Covariate Effects on Latent Class 1............................................................................................................... 60 Table 9. Study Factors with Medium Effect Sizes on Standard Error Biases of Conditional Response Probabilities of Latent Class 1 ..................................................................................... 70 Table 10. Study Factors with Strong Effect Sizes on 95% Confidence Interval Coverage Rate of 
Level-1 Covariate Effects on the Latent Class 1 ........................................................................... 80 Table 11. Study Factors with Strong Effect Sizes on 95% Confidence Interval Coverage Rate of 
Level-1 Covariate Effects on the Latent Class 2 ........................................................................... 84 Table 12. Study Factors with Strong Effect Sizes on 95% Confidence Interval Coverage Rate of 
Level-2 Covariate Effect ............................................................................................................... 87 Table 13. Study Factors with Strong Effect Sizes on 95% Confidence Interval Coverage Rate of 
Conditional Response Probability of Latent Class 1 .................................................................... 91 Table 14. Study Factors with Strong Effect Sizes on 95% Confidence Interval Coverage Rate of 
Conditional Response Probability of Latent Class 2 .................................................................... 95 x  Table 15. Study Factors with Strong Effect Sizes on 95% Confidence Interval Coverage Rate of Conditional Response Probability of Latent Class 3 .................................................................... 99 Table 16.  Conditional Response Probabilities from NP-MLCA and Conditional NP-MLCA Models ......................................................................................................................................... 105 Table 17. Model Fit Information and Classification Quality ..................................................... 106 Table 18. The Level-1 Covariate Effects on Level-1 Latent Class Solution (Scale: Odds Ratio)..................................................................................................................................................... 108 Table 19. Individual-Level Classification Accuracy by Study Factors ....................................... 128 Table 20. Group-Level Classification Accuracy by Study Factors ............................................. 130 Table 21. Bias of Level-1 Covariate Effect on Latent Class 1 by Study Factors ........................ 132 Table 22. Bias of Level-1 Covariate Effect on Latent Class 2 by Study Factors ........................ 134 Table 23. Bias of Level-2 Covariate Effect by Study Factors ..................................................... 136 Table 24. Bias of Conditional Response Probability of Latent Class 1 by Study Factors ......... 138 Table 25. Bias of Conditional Response Probability of Latent Class 2 by Study Factors ......... 140 Table 26. Bias of Conditional Response Probability of Latent Class 3 by Study Factors ......... 142 Table 27. Standard Error Biases of Level-1 Covariate Effect on the Latent Class 1 by Study Factors ........................................................................................................................................ 144 Table 28. Standard Error Biases of Level-1 Covariate Effect on the Latent Class 2 by Study Factors ........................................................................................................................................ 146 Table 29. Standard Error Biases of Level-2 Covariate Effect by Study Factors ........................ 148 Table 30. Standard Error Biases of Conditional Response Probability of Latent Class 1 by Study 
Factors ........................................................................................................................................ 150 Table 31. Standard Error Biases of Conditional Response Probability of Latent Class 2 by Study 
Factors ........................................................................................................................................ 152 xi  Table 32. Standard Error Biases of Conditional Response Probability of Latent Class 3 by Study Factors ........................................................................................................................................ 154 Table 33. 95% Confidence Interval Coverage Rates of Level-1 Covariate Effect on the Latent Class 1 by Study Factors............................................................................................................. 156 Table 34. 95% Confidence Interval Coverage Rates of Level-1 Covariate Effect on the Latent Class 2 by Study Factors............................................................................................................. 158 Table 35. 95% Confidence Interval Coverage Rates of Level-2 Covariate Effect by Study Factors..................................................................................................................................................... 160 Table 36. 95% Confidence Interval Coverage Rates of Conditional Response Probability of 
Latent Class 1 by Study Factors ................................................................................................. 162 Table 37. 95% Confidence Interval Coverage Rates of Conditional Response Probability of 
Latent Class 2 by Study Factors ................................................................................................. 164 Table 38. 95% Confidence Interval Coverage Rates of Conditional Response Probability of 
Latent Class 3 by Study Factors ................................................................................................. 166      xii  LIST OF FIGURES   Figure 1. Two-way interaction of study factors on level-1 classification accuracy  a) quality of latent class indicators × overall sample size, b) number of indicators × overall sample size, and c) quality of indicators × number of indicators. ............................................................................... 34 Figure 2. Two-way interaction of study factors on level-1 classification accuracy  a) quality of latent class indicators × overall sample size, b) number of indicators × overall sample size, and c) quality of indicators × number of indicators. ............................................................................... 35 Figure 3. Two-way interaction of study factors on level-1 classification accuracy  a) level-2 covariate effect × quality of indicators, b) level-2 covariate effect × number of indicators, and c) level-2 covariate effect × overall sample size. ............................................................................. 35 Figure 4. Two-way interaction of study factors on level-1 classification accuracy: level-2 covariate effect × level-1 covariate effects. ................................................................................. 36 Figure 5. Two-way interaction of study factors on leve-2 classification accuracy a) quality of latent class indicators × overall sample size, b) number of indicators × overall sample size, and c) quality of indicators × number of indicators. ............................................................................... 37 Figure 6. Two-way interaction of study factors on level-2 classification accuracy a) level-1 covariate effects × quality of indicators , b) level-1 covariate effects × number of indicators, and c) level-1 covariate effect × overall sample size. ......................................................................... 38 Figure 7. Two-way interaction of study factors on level-2 classification accuracy  a) level-2 covariate effect × quality of indicators, b) level-2 covariate effect × number of indicators, and c) level-2 covariate effect × overall sample size. ............................................................................. 38 Figure 8. Two Œway interaction of study factors on level-2 classification accuracy: level-2 covariate effect × level-1 covariate effects. ................................................................................. 39 Figure 9. Two-way interaction of study factors on mean bias of level-1 covariate effect on latent class 1  a) quality of latent class indicators × overall sample size, b) number of indicators × overall sample size, and c) quality of indicators × number of indicators. .................................... 41 Figure 10. Two-way interaction of study factors on mean bias of level-1 covariate effect on latent class 1 a) level-1 covariate effects × quality of indicators, b) level-1 covariate effects × number of indicators, and c) level-1 covariate effect × overall sample size. ............................... 41 xiii  Figure 11. Two-way interaction of study factors on mean bias of level-1 covariate effect on latent class 1 a) level-2 covariate effect × quality of indicators, b) level-2 covariate effect × number of indicators, and c) level-2 covariate effect × overall sample size. ............................... 42 Figure 12. Two-way interaction of study factors on mean bias of level-1 covariate effect on latent class 1: level-2 covariate effect × level-1 covariate effects. ............................................... 42 Figure 13. Two-way interaction of study factors on mean bias of level-1 covariate effect on latent class 2: a) quality of latent class indicators × overall sample size, b) number of indicators × overall sample size, and c) quality of indicators × number of indicators. ................................ 43 Figure 14. Two-way interaction of study factors on mean bias of level-1 covariate effect on latent class 2: a) quality of latent class indicators × overall sample size, b) number of indicators × overall sample size, and c) quality of indicators × number of indicators. ................................ 44 Figure 15. Two-way interaction of study factors on mean bias of level-1 covariate effect on latent class 2: a) level-2 covariate effect × quality of indicators, b) level-2 covariate effect × number of indicators, and c) level-2 covariate effect × overall sample size. ............................... 44 Figure 16. Two-way interaction of study factors on mean bias of level-1 covariate effect on latent class 2: level-2 covariate effects × level-1 covariate effects. ............................................. 45 Figure 17. Two-way interaction of study factors on mean bias of level-2 covariate effect: a) quality of latent class indicators × overall sample size, b) number of indicators × overall sample size, and c) quality of indicators × number of indicators. ............................................................ 46 Figure 18. Two-way interaction of study factors on mean bias of level-2 covariate effect a) level-1 covariate effects × quality of indicators , b) level-1 covariate effects × number of indicators, and c) level-1 covariate effect × overall sample size. .................................................................. 46 Figure 19. Two-way interaction of study factors on mean bias of level-2 covariate effect a) level-2 covariate effect × quality of indicators, b) level-2 covariate effect × number of indicators, and c) level-2 covariate effect × overall sample size. ......................................................................... 47 Figure 20. Two-way interaction of study factors on mean bias of level-2 covariate effect: level-2 covariate effect × level-1 covariate effects. .................................................................................. 47 Figure 21. Two-way interaction of study factors on mean bias of conditional response probabilities of latent class 1: a) quality of latent class indicators × overall sample size, b) number of indicators × overall sample size, and c) quality of indicators × number of indicators........................................................................................................................................................ 49 xiv  Figure 22. Two-way interaction of study factors on mean bias of conditional response probabilities of latent class 1: a) level-1 covariate effects × quality of indicators, b) level-1 covariate effects × number of indicators, and c) level-1 covariate effect × overall sample size. 49 Figure 23. Two- way interaction of study factors on mean bias of conditional response probabilities of latent class 1: a) level-2 covariate effect × quality of indicators, b) level-2 covariate effect × number of indicators, and c) level-2 covariate effect × overall sample size. . 50 Figure 24. Two-way interaction of study factors on mean bias of conditional response probabilities of latent class 1: level-2 covariate effects × level-1 covariate effects. .................... 50 Figure 25. Two-way interaction of study factors on mean bias of conditional response probabilities of latent class 2: a) quality of latent class indicators × overall sample size, b) number of indicators × overall sample size, and c) quality of indicators × number of indicators........................................................................................................................................................ 52 Figure 26. Two-way interaction of study factors on mean bias of conditional response probabilities of latent class 2: a) level-1 covariate effects × quality of indicators , b) level-1 covariate effects × number of indicators, and c) level-1 covariate effect × overall sample size. 52 Figure 27. Two-way interaction of study factors on mean bias of conditional response probabilities of latent class 2: a) level-2 covariate effect × quality of indicators, b) level-2 covariate effect × number of indicators, and c) level-2 covariate effect × overall sample size. . 53 Figure 28. Two-way interaction of study factors on mean bias of conditional response probabilities of latent class 2: level-2 covariate effects × level-1 covariate effects. .................... 53 Figure 29. Two-way interaction of study factors on mean bias of conditional response probabilities of latent class 3: a) quality of latent class indicators × overall sample size, b) number of indicators × overall sample size, and c) quality of indicators × number of indicators........................................................................................................................................................ 55 Figure 30. Two-way interaction of study factors on mean bias of level-2 covariate effect a) level-1 covariate effects × quality of indicators , b) level-1 covariate effects × number of indicators, and c) level-1 covariate effect × overall sample size. .................................................................. 55 Figure 31. Two-way interaction of study factors on mean bias of conditional response probabilities of latent class 3: a) level-2 covariate effect × quality of indicators, b) level-2 covariate effect × number of indicators, and c) level-2 covariate effect × overall sample size. . 56 Figure 32. Two-way interaction of study factors on mean bias of conditional response probabilities of latent class 3: level-2 covariate effects × level-1 covariate effects. .................... 56 xv  Figure 33. Two-way interaction of study factors on mean standard error estimate bias of level-1 covariate effects on the latent class 1: a) quality of latent class indicators × overall sample size, b) number of indicators × overall sample size, and c) quality of indicators × number of indicators........................................................................................................................................................ 58 Figure 34. Two-way interaction of study factors on mean standard error estimate bias of level-1 covariate effects on the latent class 1 a) level-1 covariate effects × quality of indicators, b) level-1 covariate effects × number of indicators, and c) level-1 covariate effect × overall sample size........................................................................................................................................................ 58 Figure 35. Two-way interaction of study factors on mean standard error estimate bias of level-1 covariate effects on the latent class 1: a) level-2 covariate effect × quality of indicators, b) level-2 covariate effect × number of indicators, and c) level-2 covariate effect × overall sample size........................................................................................................................................................ 59 Figure 36. Two-way interaction of study factors on mean standard error estimate bias of level-1 covariate effects on the latent class 1: level-2 covariate effects × level-1 covariate effects. ....... 59 Figure 37. Two-way interaction of study factors on mean standard error estimate bias of level-1 covariate effects on the latent class 2: a) quality of latent class indicators × overall sample size, b) number of indicators × overall sample size, and c) quality of indicators × number of indicators........................................................................................................................................................ 61 Figure 38. Two-way interaction of study factors on mean standard error estimate bias of level-1 covariate effects on the latent class 2: a) level-1 covariate effects × quality of indicators, b) level-1 covariate effects × number of indicators, and c) level-1 covariate effect × overall sample size. ............................................................................................................................................... 62 Figure 39. Two-way interaction of study factors on mean standard error estimate bias of level-1 covariate effects on the latent class 2: a) level-2 covariate effect × quality of indicators, b) level-2 covariate effect × number of indicators, and c) level-2 covariate effect × overall sample size........................................................................................................................................................ 63 Figure 40. Two-way interaction of study factors on mean standard error estimate bias of level-1 covariate effects on the latent class 2: level-2 covariate effects × level-1 covariate effects. ....... 63 Figure 41. Two-way interaction of study factors on mean standard error estimate bias of level-2 covariate effect: a) quality of latent class indicators × overall sample size, b) number of indicators × overall sample size, and c) quality of indicators × number of indicators. ............... 65 Figure 42. Two-way interaction of study factors on mean standard error estimate bias of level-2 covariate effect: a) level-1 covariate effects × quality of indicators , b) level-1 covariate effects × number of indicators, and c) level-1 covariate effect × overall sample size. ........................... 65 xvi  Figure 43. Two-way interaction of study factors on mean standard error estimate bias of level-2 covariate effect: a) level-2 covariate effect × quality of indicators, b) level-2 covariate effect × number of indicators, and c) level-2 covariate effect × overall sample size. ............................... 66 Figure 44. Two-way interaction of study factors on mean standard error estimate bias of level-2 covariate effect: level-2 covariate effects × level-1 covariate effects. ......................................... 66 Figure 45. Two-way interaction of study factors on mean standard error estimate bias of conditional response probabilities of latent class 1: a) quality of latent class indicators × overall sample size, b) number of indicators × overall sample size, and c) quality of indicators × number of indicators. ................................................................................................................................. 67 Figure 46. Two-way interaction of study factors on mean standard error estimate bias of conditional response probabilities of latent class 1: a) level-1 covariate effects × quality of indicators , b) level-1 covariate effects × number of indicators, and c) level-1 covariate effect × overall sample size. ....................................................................................................................... 68 Figure 47. Two-way interaction of study factors on mean standard error estimate bias of conditional response probabilities of latent class 1: a) level-2 covariate effect × quality of indicators, b) level-2 covariate effect × number of indicators, and c) level-2 covariate effect × overall sample size. ....................................................................................................................... 69 Figure 48. Two-way interaction of study factors on mean standard error estimate bias of conditional response probabilities of latent class 1: level-2 covariate effects × level-1 covariate effects. ........................................................................................................................................... 70 Figure 49. Two-way interaction of study factors on mean standard error estimate bias of conditional response probabilities of latent class 2: a) quality of latent class indicators × overall sample size, b) number of indicators × overall sample size, and c) quality of indicators × number of indicators. ................................................................................................................................. 71 Figure 50. Two-way interaction of study factors on mean standard error estimate bias of conditional response probabilities of latent class 2: a) level-1 covariate effects × quality of indicators, b) level-1 covariate effects × number of indicators, and c) level-1 covariate effect × overall sample size. ....................................................................................................................... 72 Figure 51. Two-way interaction of study factors on mean standard error estimate bias of conditional response probabilities of latent class 2: a) level-2 covariate effect × quality of indicators, b) level-2 covariate effect × number of indicators, and c) level-2 covariate effect × overall sample size. ....................................................................................................................... 73 xvii  Figure 52. Two-way interaction of study factors on mean standard error estimate bias of conditional response probabilities of latent class 2: level-2 covariate effects × level-1 covariate effects. ........................................................................................................................................... 73 Figure 53. Two-way interaction of study factors on mean standard error estimate bias of conditional response probabilities of latent class 3: a) quality of latent class indicators × overall sample size, b) number of indicators × overall sample size, and c) quality of indicators × number of indicators. ................................................................................................................................. 74 Figure 54. Two-way interaction of study factors on mean standard error estimate bias of conditional response probabilities of latent class 3: a) level-1 covariate effects × quality of indicators, b) level-1 covariate effects × number of indicators, and c) level-1 covariate effect × overall sample size. ....................................................................................................................... 75 Figure 55. Two-way interaction of study factors on mean standard error estimate bias of conditional response probabilities of latent class 3: a) level-2 covariate effect × quality of indicators, b) level-2 covariate effect × number of indicators, and c) level-2 covariate effect × overall sample size. ....................................................................................................................... 76 Figure 56. Two-way interaction of study factors on mean standard error estimate bias of conditional response probabilities of latent class 3: level-2 covariate effects × level-1 covariate effects. ........................................................................................................................................... 76 Figure 57. Two-way interaction of study factors on mean 95% confidence interval coverage rates of level-1 covariate effects on the latent class 1: a) quality of latent class indicators × overall sample size, b) number of indicators × overall sample size, and c) quality of indicators × number of indicators. ................................................................................................................................. 77 Figure 58. Two-way interaction of study factors on mean 95% confidence interval coverage rates of level-1 covariate effects on the latent class 1 a) level-1 covariate effects × quality of indicators, b) level-1 covariate effects × number of indicators, and c) level-1 covariate effect × overall sample size. ................................................................................................................................... 78 Figure 59. Two-way interaction of study factors on mean 95% confidence interval coverage rates of level-1 covariate effects on the latent class 1: a) level-2 covariate effect × quality of indicators, b) level-2 covariate effect × number of indicators, and c) level-2 covariate effect × overall sample size. ................................................................................................................................... 79 Figure 60. Two-way interaction of study factors on mean 95% confidence interval coverage rates of level-1 covariate effects on the latent class 1: level-2 covariate effects × level-1 covariate effects. ........................................................................................................................................... 79 xviii  Figure 61. Two-way interaction of study factors on mean 95% confidence interval coverage rates of level-1 covariate effects on the latent class 2: a) quality of latent class indicators × overall sample size, b) number of indicators × overall sample size, and c) quality of indicators × number of indicators. ................................................................................................................................. 81 Figure 62. Two-way interaction of study factors on mean 95% confidence interval coverage rates of level-1 covariate effects on the latent class 2: a) level-1 covariate effects × quality of indicators, b) level-1 covariate effects × number of indicators, and c) level-1 covariate effect × overall sample size. ....................................................................................................................... 82 Figure 63. Two-way interaction of study factors on mean 95% confidence interval coverage rates of level-1 covariate effects on the latent class 2: a) level-2 covariate effect × quality of indicators, b) level-2 covariate effect × number of indicators, and c) level-2 covariate effect × overall sample size. ................................................................................................................................... 83 Figure 64. Two-way interaction of study factors on mean 95% confidence interval coverage rates of level-1 covariate effects on the latent class 2: level-2 covariate effects × level-1 covariate effects. ........................................................................................................................................... 83 Figure 65. Two-way interaction of study factors on mean 95% confidence interval coverage rates of level-2 covariate effects: a) quality of latent class indicators × overall sample size, b) number of indicators × overall sample size, and c) quality of indicators × number of indicators. ........... 85 Figure 66. Two-way interaction of study factors on mean 95% confidence interval coverage rates of level-2 covariate effects: a) level-1 covariate Effects × quality of indicators, b) level-1 covariate effects × number of indicators, and c) level-1 covariate effect × overall sample size. 85 Figure 67. Two-way interaction of study factors on mean 95% confidence interval coverage rates of level-2 covariate effects: a) level-2 covariate effect × quality of indicators, b) level-2 covariate effect × number of indicators, and c) level-2 covariate effect × overall sample size. ................. 86 Figure 68. Two-way interaction of study factors on mean 95% confidence interval coverage rates of level-2 covariate effects: level-2 covariate effects × level-1 covariate effects. ....................... 86 Figure 69. Two-way interaction of study factors on mean 95% confidence interval coverage rates of conditional response probabilities of latent class 1: a) quality of latent class indicators × overall sample size, b) number of indicators × overall sample size, and c) quality of indicators × number of indicators. .................................................................................................................... 88 Figure 70. Two-way interaction of study factors on mean 95% confidence interval coverage rates of conditional response probabilities of latent class 1: a) level-1 covariate Effects × quality of indicators, b) level-1 covariate effects × number of indicators, and c) level-1 covariate effect × overall sample size. ....................................................................................................................... 89 xix  Figure 71. Two-way interaction of study factors on mean 95% confidence interval coverage rates of conditional response probabilities of latent class 1: a) level-2 covariate effect × quality of indicators, b) level-2 covariate effect × number of indicators, and c) level-2 covariate effect × overall sample size. ....................................................................................................................... 90 Figure 72. Two-way interaction of study factors on mean 95% confidence interval coverage rates of conditional response probabilities of latent class 1: level-2 covariate effects × level-1 covariate effects. ........................................................................................................................... 90 Figure 73. Two-way interaction of study factors on mean 95% confidence interval coverage rates of conditional response probabilities of latent class 2: a) quality of latent class indicators × overall sample size, b) number of indicators × overall sample size, and c) quality of indicators × number of indicators. .................................................................................................................... 92 Figure 74. Two-way interaction of study factors on mean 95% confidence interval coverage rates of conditional response probabilities of latent class 2: a) level-1 covariate effects × quality of indicators, b) level-1 covariate effects × number of indicators, and c) level-1 covariate effect × overall sample size. ....................................................................................................................... 93 Figure 75. Two-way interaction of study factors on mean 95% confidence interval coverage rates of conditional response probabilities of latent class 2: a) level-2 covariate effect × quality of indicators, b) level-2 covariate effect × number of indicators, and c) level-2 covariate effect × overall sample size. ....................................................................................................................... 94 Figure 76. Two-way interaction of study factors on mean 95% confidence interval coverage rates of conditional response probabilities of latent class 2: level-2 covariate effects × level-1 covariate effects. ........................................................................................................................... 94 Figure 77. Two-way interaction of study factors on mean 95% confidence interval coverage rates of conditional response probabilities of latent class 3: a) quality of latent class indicators × overall sample size, b) number of indicators × overall sample size, and c) quality of indicators × number of indicators. .................................................................................................................... 96 Figure 78. Two-way interaction of study factors on mean 95% confidence interval coverage rates of conditional response probabilities of latent class 3: a) level-1 covariate Effects × quality of indicators, b) level-1 covariate effects × number of indicators, and c) level-1 covariate effect × overall sample size. ....................................................................................................................... 97 Figure 79. Two-way interaction of study factors on mean 95% confidence interval coverage rates of conditional response probabilities of latent class 3: a) level-2 covariate effect × quality of indicators, b) level-2 covariate effect × number of indicators, and c) level-2 covariate effect × overall sample size. ....................................................................................................................... 98 xx  Figure 80. Two-way interaction of study factors on mean 95% confidence interval coverage rates of conditional response probabilities of latent class 3: level-2 covariate effects × level-1 covariate effects. ........................................................................................................................... 98 Figure 81. The bar plot of ten observed indicators (measures) before dichotomization. ........... 102 Figure 82. The bar plot of ten observed indicators (measures) after dichotomization. .............. 103 Figure 83. The conditional response probability patterns on ten observed indicators in NP-MLCA model. ............................................................................................................................. 106 Figure 84. The composition of site-level latent classes in NP-MLCA model............................ 107 Figure 85. The composition of site-level latent classes in conditional NP-MLCA model......... 107  1  Chapter 1   Introduction Program evaluators have long been interested in finding rigorous methods to identify better performed programs and successful program participants, and for helping underperforming participants to improve. The advantage of nonparametric multilevel latent class analysis (NP-MLCA) is that it can simultaneously identify typologies of students within schools and of the schools themselves. Similarly, it can be utilized to classify students and classes, staffs and 
companies, and patients and hospitals. It has been applied to diagnose students™ learning 
problems in subjects (e.g. mathematics or science) by identifying learning topics that require the most improvement. While covariates in the nested structured data are taken into account in the model (i.e., conditional NP-MLCA), the question of how the effects of covariates and the quality of indicators affect classification accuracy (at both the individual and group levels) still remains unanswered. The current research aims to advance our knowledge of NP-MLCA™s classification performance in cases where numerous indicators and covariates are available. When classification is the primary interest of the researcher, cluster analysis has traditionally been applied. It uses distance measures, such as nearest-neighbor classifiers, to identify clusters among the measured or observed variables (i.e., indicators). However, several limitations of this method restrict its application. First, the clusters it identifies differ greatly depending on the distance measures selected, leading the method to be applied mainly to 
exploratory purposes. Second, the way the observed indicators are ordered also has a profound impact on clustering results. Third, if a case is dropped, the clustering changes drasticallyΠa further indication of the oversensitivity of the method. Fourth, the hierarchy of a clustering 
solution is not easily rectified. In short, the results of clustering are often sensitive and unstable 2  due to their reliance on distance measures, and for this reason, a more satisfactory solution is necessary.  Moreover, classification error cannot be incorporated into subsequent analyses of the clusters that the method produces. If researchers want to further investigate the characteristics of 
the clusters by examining the relationship between covariates and the cluster variable, or by 
examining the effect of clusters on an outcome variable (such as achievement), the usual strategy is to treat the estimated cluster membership as the observed variable, and conduct a relational 
analysis as the second step. But again, because different distance measures can yield very 
different clustering solutions, the results of subsequent analyses are likely to result in widely divergent statistical inferences. Multiple hypothesis testing is a concern in classification methods. A conventional procedure for classifying students in a nested structure is to conduct standard latent class analysis for each school. However, this approach results in inflated Type I errors. Even though a Bonferroni correction or familywise error rate can be applied, such ad hoc corrections are either too liberal or too conservative. By simultaneously classifying individual-level units and higher-level units, NP-MLCA addresses the shortcomings of ad hoc approaches to controlling the Type I error rate. Traditionally, when the higher-level classification of multilevel data is the main interest of a study, researchers aggregate the information from the lower level to the higher level. For 
example, school-specific achievement (i.e., school means) are aggregated from students™ individual scores for the purpose of school evaluation, a predetermined proportion or cut-off score is utilized, above which schools are deemed better performed. The shortcomings of this type of aggregation approach are as follows. First, the measure of central tendency may not be 3  the best way to represent the school, especially when there are extreme values (i.e., outliers) or high variability in the school data. Second, aggregation obscures the influence of the individual levels, leading to ecological fallacy (Robinson, 1950). Third, aggregation results in a reduced sample size (i.e., the number of higher-level units is always smaller than the number of lower-level ones), which substantially reduces statistical power. As such, the development of a method 
that can address these issues would be highly beneficial. NP-MLCA is a method for identifying latent classes (i.e., human subgroups or types that are not identifiable except via their pattern of responses to a given set of indicators) at multiple levels simultaneously. The higher-level classification in NP-MLCA is based on which patterns of latent classes (e.g., smokers, bullying victims) are prevalent in each higher-level unit (e.g., school). Schools are therefore more likely to be classified into the same group with other schools 
that have similar distribution of individual-level latent-class patterns. Because NP-MLCA is a model-based technique using probability models, model-fit indices can be utilized to identify optimal solutions, in terms of the number of latent classes (i.e., mixture components). The 
probability of a person endorsing the response of the observed indicators can be estimated, and 
so can the latent class probability (also known as latent-class prevalence or mixture weights). Since all latent classes are identified simultaneously, the statistical power problems caused by 
reduced sample sizes can also be improved. Another advantage of NP-MLCA™s model-based approach is that it provides researchers with the flexibility to incorporate covariates into the model to contextualize latent classes while identification of those classes is still ongoing 
(sometimes known as a one-step procedure). Classification error can thus be taken into account during the iteration process of estimation, when the relationship of interest is that between the 
covariates and the latent-class variables. 4  Covariate-incorporating strategies in NP-MLCA have gone largely unexamined; however, one-step and three-step strategies have commonly been studied in LCA. It is important to know whether these strategies are suitable for use with multilevel structured data, and therefore with NP-MLCA models. The covariate-conditioned posterior probability used in the one-step approach for identifying latent class membership is analogous to the multiple-indicator/multiple-cause model developed in factor analysis (Vermunt, 2010), while the main purpose of adopting three-step procedures (i.e., the classify-analyze method) for LCA (Clogg, 1995) is to contextualize the latent class. Although the purposes for using them are different, previous studies have shown that the one-step strategy produces unbiased and efficient parameter estimates, whereas the three-step strategy results require bias correction for which various methods have been proposed (Bakk, Tekle, & Vermunt, 2013; Bolck, Croon, & Hagenaars, 
2004). There have been abundant simulation studies investigating covariate-incorporating LCA in terms of the performance of the covariate effect as well as the performance of parameter estimations in different sample size scenarios (e.g., Bandeen-Roche, Miglioretti, Zeger, & Rathouz, 1997; Vermunt, 2010; Wurpts & Geiser, 2014), but similar studies in multilevel contexts have been very few (i.e., Finch & French, 2013; Henry & Muthén, 2010; Muthén & Asparouhov, 2009). 1.1 Research Goal Although NP-MLCA has been widely applied in multiple disciplines, two major issues confront its use in multilevel contexts: how to accurately classify individual-level and group-level units, and how to provide precise model parameter estimations when covariates are 
included in the model. Therefore, the proposed research intends to answer how the quality of two outcome measures Πparameter estimation and classifcation accuracies Πare affected by the 5  following five study factors at both levels in NP-MLCA: 1) the quality of latent class indicators (i.e., conditional response parameters), 2) the number of latent class indicators, 3) the effect size of covariates at different levels, 4) the number of groups/higher-level units (e.g., schools, hospitals, companies), and 5) the size of groups/higher-level units (e.g., the number of students within each school, the number of patients within each hospital, and the number of personnel within each company). At the end of the proposed simulation-based investigation, the performance of the two primary outcome measures will be examined via factorial-designed analyses of variance (ANOVA). Specifically, classification accuracy is measured via proportion consistency between true latent class membership and the predicted latent class membership at each level, while the quality of parameter estimation is evaluated in terms of bias, standard error bias, and the 95% confidence interval coverage rate.    6  Chapter 2   Latent Class Analysis and Multilevel Latent Class Analysis As a person-centered method, latent class analysis (LCA, Lazarsfeld & Henry, 1968) focuses on grouping subjects on the basis of their response patterns to a set of indicators. Under the framework of structural equation modeling, a categorical latent variable is utilized to capture 
the relationships among measured indicators and to represent the subjects™ class membership. The following sections briefly introduce the standard LCA model, model assumptions and 
practical modeling strategies that have emerged from previous research. More detailed 
descriptions of this model can be found in Collins and Lanza (2010) and Wang and Wang (2012), and of finite mixture modeling in Everitt & Hand (1981), Mclachlan & Peel, (2000),  Schlattmann (2009), and Titterington, Smith, & Makov (1985).  2.1 Latent Class Analysis 2.1.1 Model Specification The probability of observing a subject i responding to K indicators (e.g., items, observed measures) with a particular response pattern s, (=), is a function of two components: a) the probability of a person choosing a particular response to an indicator, conditional on the subject™s latent class membership, and b) the probability of the person being in a specific latent class. Standard LCA can be expressed as follows:  (=)=(=)(=|=) (1) here  is the k-tuple response pattern of subject i to K indicators;  is a k-tuple vector of a specific response pattern;   is the item response of subject i for indicator k;  is the particular 7  response for indicator k;  = 1, 2, –; and  is the number of response levels that item k has. If indicator k is dichotomous, then  is equal to 2.  denotes the latent class variable, and the latent class membership c = 1, 2, –L, with L denoting the total number of the latent classes. In other words, the probability of observing a particular response pattern is a weighted average of item response probabilities conditional on their latent class probability. The first component on the right-hand side of equation (1) is the latent class probability, denoted by (=). The second component on the same side of the equation refers to conditional response probabilities (CRP) (also called conditional item response probabilities or class-specific probabilities). Since it is class-specific (i.e., conditional on latent class membership), the CRP is utilized to label the latent classes. 2.1.1.1 Local Independence The primary assumption of LCA is that the indicators  are statistically independent within a given latent class (Clogg, 1995; McCutcheon, 1987; Reboussin & Anthony, 2001; Vermunt & Magidson, 2002). It is analogous to the use of factors in confirmatory factor analysis (CFA), CFA utilizes the latent variable to explain the relationships among the measured indicators; however, in LCA, the latent variable is a categorical one. In other words, it is assumed that the latent variable accounts for the observed relationships among the indicators. Based on the assumption of local independence, CRP can be calculated as the product of the particular 
responses to each indicator:  (=|=)=(=|=)  (2) Thus, (1) can be written as follows:  8   (=)=(=)(=|=)  (3) 2.1.1.2 Latent Class Probability Also called latent class prevalence or posterior class probability, latent class probability here in equation (3) is unconditional and denotes the probability that subject i is a member of a particular class. For each subject, (L-1) latent class probabilities are estimated, and the summation of the probability that a subject belongs to each latent class should be 1, (=)=1. Conventionally, the largest estimated probability is used to define the subject™s latent class membership. When the population size is known, the estimated latent class size can be calculated by multiplying population size by estimated latent class probability. When L is larger than two, latent class probability is as follows,  (=)=exp()exp() (4) with  being the log odds of belonging to class c. For identification, one constraint has to be imposed on one of logit parameters, for example, =0.  2.1.1.3 Conditional Response Probability (CRP) Also called item response parameter, CRP is the probability of an individual selecting response  to item  given his/her latent class membership (i.e., it is class-specific). It can be written as,   (=|=)=expexp (5) with  denoting the log odds of a person choosing response  given his/her latent class membership of class c. Similarly to latent class probability, one of the logit parameters needs to 9  be constrained for identification, for example, =0. If we substitute equation (4) and (5) into (3), then (1) can be formulated as,    Since conditional response probabilities also represents the strength of the relationship between the latent class indicators and the latent class variable. The higher the CRP of the indicator is, the more ideal the quality of indicators. Therefore, CRP and quality of indicators are used interchangeably in the literature. 2.2 Nonparametric Multilevel Latent Class Analysis  Applying traditional LCA in multilevel data produces biased estimation as the intraclass correlation increases and sample sizes decrease (Kaplan & Keller, 2011). In addition, when there are more than a few groups, it is impractical to estimate separate sets of parameters for the groups as in multiple group LCA. Unlike traditional LCA, which fixes the latent class 
probabilities at the same level as one another, MLCA allows latent class probabilities and CPR to 
vary from group to group. In MLCA, in other words, group-specific parameters are introduced into equations (1) through (6), and random components are also added in to capture the 
variations in each group™s latent class probabilities and CRPs. For J groups, there are J sets of latent class probabilities and CRPs to estimate. More detailed descriptions of this model can be found in Vermunt (2003).  (=)=exp()exp()expexp  (6) 10  2.2.1 Model Specification Let subject i = 1, 2, –, , and group =1, 2, –, .  is the response pattern to  indicators of subject  in group , and  is a vector of a specific response pattern.   is the item response for subject  in group  to indicator .  denotes the latent class variable at the person level, and the individual-level latent class membership  = 1, 2, –, , where  denotes the total number of the individual-level latent classes. Therefore, as the local independence assumption holds, by introducing group index  into equation (3), MLCA model can be specified as follows:  ===== (7)                                       = == = (8) Similarly to the probability of the response patterns, latent class probabilities can be specified using the following equations.  ==expexp (9) Group-specific CRPs can also be specified by introducing index j in equation (6),  = ==expexp (10) If we substitute equation (9) and (10) into (8), it yields equation (11):  = =expexp expexp  (11) 11  For model identification, the random intercepts   in (9) and  in (10) are set to be 0. Equation (10) represents group-specific CRP, allowing for the fact that individuals in different groups can respond to indicators in different ways. This feature can be used to examine how items function differently across groups (i.e., DIF). For the sake of the simplicity of the model used in this dissertation, CRPs are assumed to be the same across groups (i.e., assuming invariant measurement error). Therefore, by applying (5) instead of (10), equation (11) can be simplified 
as follows, which is the MLCA model without covariates (i.e., null model):  = =expexp expexp (12) In parametric MLCA, random intercepts  have an underlying distribution assumption. By adopting a random component in , the group-specific effects can be obtained, and specified as follows,  =+ (13) where  denotes the random component (i.e., group-specific effects); the parametric approach traditionally assumes that ~(0,1) distribution. Equation (13) represents the assumption that the between-group variation in the log odds of belonging to the latent class c, rather than the reference latent class, follows a Normal distribution with mean  and standard deviation . Since there are  latent classes identified at the individual level, (1) random intercepts  need to be specified. Similarly, (1) independent intraclass correlations can be obtained as follows:  =+3 (14) 12   Rather than restricting all the random intercepts  to a scale continuum and making a strong assumption that the random component () is normally distributed, NP-MLCA utilizes a discrete unspecified mixing distribution for the random component. In other words, the assumption of Normal distribution is replaced with the Dirichlet distribution (i.e., the multinomial distribution), and the higher-level units in the same mixture component shares the same parameter values. The mixture components are therefore deemed as the higher-level latent classes.  Following the notation used in Vermunt (2003),  denotes a particular mixture component,  ===exp()exp()  (15) with  denoting the higher-level latent class variable that the Pth group belongs to, and is under a discrete mixing distribution. Since the groups belong to the same higher-level latent class and share the same parameter values, the random component in NP-MLCA is not group specific (with  subscript) as in equation (9), but specific to the higher-level latent class, as in equation (15). Group-level latent class membership  = 1, 2, –, , where  denotes the total number of the group-level latent classes, which is also the number of mass points in the discrete mixing distribution. Similar to the parametric approach, the random intercepts () from the individual-level latent class can be represented as follows,  =+ (16) where  denotes the random component (i.e., higher-level latent-class-specific effects). For example, if =2, the random intercepts of all groups can be classified into two classes, indicating that  follows the Binomial distribution. If >2, the random intercepts of all 13  groups can be classified into more than three classes, showing that  follows the Dirichlet (multinomial) distribution. 2.2.2 Conditional Independence Assumption Analogous to the local independence assumption in LCA, in multilevel contexts, it is assumed that the individual™s responses are independent of each other given their group-level and individual-level latent class membership.  Another assumption generally seen in the multilevel context is that the nested effect is captured by random components. That is, controlling for the random effect (i.e., random components), observations of individuals are mutually independent. These two assumptions will be applied in Chapter 3, below.  The fit indices in NP-MLCA™s procedure for deciding the number of lower-level latent classes and higher-level latent classes have not yet been thoroughly studied, with the exception of the performance criteria: Bayesian Information Criterion (BIC, Schwarz, 1978), Akaike Information Criterion (AIC, Akaike, 1974), consistent AIC (CAIC, Bozdogan, 1987) and AIC3 (Bozdogan, 1993) in . In the present dissertation, based on the above-cited studies™ results and for the sake of simplicity, we will use BIC and AIC to check model fit. The number of lower-level latent classes will be fixed at three, and the number of higher-level latent classes at two. The number of response levels for each item is fixed at two (i.e., binary 
indicators) across all indicators that we propose to use. 14  2.3 Adding Covariates in Nonparametric Multilevel Latent Class Analysis By incorporating covariates into NP-MLCA, individual-level latent class probability becomes conditional, i.e., a function of those covariates, as shown in equation (17): with X denoting the lower-level covariates, and  denoting the effect (i.e., slope) of the covariate X.   =,,==exp +exp + (17) The random intercepts  can be formulated as:   =++ (18) Similarly,  denotes the higher-level covariate, and  denotes the effect (i.e., slope) of the covariate . Equation (18) also allows the covariate effect to vary along with the higher-level mixtures. Previous literature regarding the incorporation of covariates into MLCA is rare, though two methods in LCA, the one-step method and the three-step method, have been investigated (Bakk et al., 2013; Bray, Lanza, & Tan, 2015; Vermunt, 2010). The first of these estimates covariate effects and identifies latent classes simultaneously using a single model. The second identifies the latent class first; treats the estimated latent class membership as the observed class 
membership; and then analyzes the relationships between the covariates and the latent class variable. Although both methods can successfully incorporate covariates into LCA models, both 
have their limitations. In the one-step method Πas demonstrated in equation (17) in the case of conditional NP-MLCA Πthe estimated probability of the latent class is a function of the covariates, so the latent class probability becomes a conditional probability. As such, different 
covariates may result in different latent class solutions, and the selection of covariates therefore plays a key role in modeling. In the three-step method, latent class probability remains 15  unconditional, but the possibility of classification error is not considered when the relationship between the covariates and the latent class variable is investigated. Therefore, the choice of a latent class assignment method becomes critically important: results using highest posterior 
probability to arrive at the final class membership assignment solution will be different from 
those using multiple pseudoclass assignment, i.e., yield differing levels of the relationship of 
covariates with the latent class variable. Simulation studies in LCA indicated that the one-step method was better when it came to parameter recovery, while the three-step method resulted in systematic errors in parameter estimation, requiring correction formulas for unbiased estimates (Bolck et al., 2004). In NP-MLCA, very few papers have hitherto discussed how model parameter estimates can be 
performed when these two methods are applied, when covariates are incorporated, and in multilevel contexts. Finch and French (2013) indicated that the parameter recovery rate of the covariate effect and the quality of indicators were both good when the covariate was incorporated into NP-MLCA. However, their study fixed the association between the covariate and the latent class variable at one, and fixed the quality of indicators at 0.8, providing us with no clues as to what differences might arise from varying covariate effects or varying indicator-quality levels. Previous literature demonstrated that high-quality indicators were beneficial to parameter estimation, and could compensate for the small effects of the covariates; however, the studies in question were focused on continuous latent variables (Marsh, Hau, Balla, & Grayson, 
1998) or standard LCA models (Wurpts & Geiser, 2014). This dissertation therefore investigates the one-step method in a range of NP-MLCA contexts. In addition, for the sake of simplicity,  will be the same across the higher-level 16  latent classes. That is, in the model specified in this dissertation, the random component of the latent class probabilities will only come from the residual components .   17  Chapter 3   Model Estimation: The Use of the Expectation-Maximum (EM) Algorithm in NP-MLCA  Due to the fact that the lower-level and higher-level latent class memberships of each subject in NP-MLCA need to be estimated, the latent class variables are missing and the dataset incomplete. The use of EM algorithms is a popular method for tackling the incomplete-data problem. Accordingly, in the present dissertation, the upward-downward Expectation Maximum algorithm (Vermunt, 2003) will be used to obtain the parameter estimates. After the construction of the loglikelihood function, two steps in the EM algorithm Œ known as the E-step and the M-step Œ are iteratively evaluated and applied to find the estimate that can maximize the likelihood function. This section briefly introduces the likelihood function and the EM algorithm for estimating model parameters in NP-MLCA, as described in greater detail by Vermunt (2003, 2008). 3.1 The Likelihood Function  The to-be-maximized loglikelihood function is constructed based on the probability density function, (|), and is used to describe the parameters in the population. Because in a multilevel context individuals are not independent of each other in the same higher-level unit, but the higher units are assumed independent, the data pertaining to higher-level units is used to construct this function.  logL=log  (19) Given the local independence assumption, with  denoting the random component and  the set unknown parameters, the higher-level probability density function is written as follows: 18         =,,,(|)  (20) There are several ways to evaluate the right-hand side of the integral part of equation (20). One of the methods is to replace the integral with a summation with M quadrature points. The integral can be accurately approximated, when the number of quadrature points increases.   =  ,,  (21) Using summation rather than integration, this method is similar to the nonparametric approach with M components, with  denoting the unknown random coefficient under a discrete mixing distribution, and  the size of each mixture component as a weighted variable, which is also an unknown parameter.  When the random component is a continuous variable, more quadrature points can better approximate the area under the density curve. However, it is assumed that the distribution of the higher level units is continuous. More quadrature points (i.e., more level-2 latent classes) may result in better model fit. It is the reason that compared with parametric MLCA, NP-MLCA is always not the best-fitting model, particularly when the random component at the higher level is continuous or when the parametric MLCA is used to generate the data.  If the higher-level units are generated from a discrete variable, and assuming that higher-level units within the same mixture components share the same value of the unknown parameters, then the discrete random component makes numerical integration unnecessary. In this case, there will be only × latent classes in total that must be identified. However, this situation requires researchers to have strong theoretical knowledge to justify the existence of mixture components on the higher level. Note that the small separation between the higher-level mixture components 19  may lead to inaccurate classification results at the higher level in some instances. In this dissertation, M is fixed at two due to the nature of the simulated data structure; note that in a real application, researchers may not know the number of groups M.  Given the local independence assumption regarding individuals™ observations, equation (21) can be rewritten as,   =  ,,, (22) Combining the probability distribution function at the higher level above and the CRP at the lower level, as in equation (8), the probability distribution function of NP-MLCA in this dissertation can be expressed as:     | ,=  (=)==,,= = (23) 3.2 EM Algorithm for Parameter Estimation The upward-downward EM algorithm (Vermunt, 2003) which will be used to obtain the estimates in NP-MLCA is very similar to the forward-backward algorithm in latent transition modeling, which is applied to the scenario when more than two categorical latent variables are involved in the model. In this dissertation, the latent class probability, =,, and the marginal posterior probability, =,=,, are obtained iteratively in the upward and downward procedures in the E-step. 20  In the E-step, the computation of the expected value of the complete data log likelihood involves calculating the marginal posterior probabilities. The marginal posterior probabilities can be specified as follows:  =,=,==, (=|=,,) (24)          =(=) ,,(=) ,, ×=, , , , , (25) Where =, is the posterior probability that group j belongs to group-level latent class m. (=|=,,) is the posterior probability that individual i belongs to individual-level latent class c given his/her group belongs to m class, conditional on the parameter estimates. The product of these two posterior probabilities gives the marginal posterior probabilities.   In the M-step, the model parameters are approximated by maximizing the loglikelihood function of the expected complete data from the E-step. The approximated estimates from the M-step are again used in the E-step to update the marginal posterior probability and the latent class probability. The iteration process between the E-step and the M-step is continued until the updated estimates™ differences meet the convergence criteria. Essentially, this estimation process assigns individuals into individual-level latent classes and simultaneously assigns the groups into higher-level latent classes.  Depending on the success of convergence, we may consider a variety of numerical integration techniques for optimization, such as trapezoid-rule, Gaussian-Hermit quadrature, or Monte Carlo integration methods. To obtain estimated asymptotic standard errors, the matrix of second-order derivatives of the log-likelihood function toward model parameters (i.e., the observed Fisher information matrix) is computed. The estimated variance-covariance matrix is 21  obtained from the inverse of the observed Fisher information matrix. (Muthén & Muthén, 1998-2015).   22  Chapter 4   Simulation Design and Outcome Measures 4.1 Simulation Design To answer the research questions, a simulation study was conducted using NP-MLCA models under 384 conditions. Data for the NP-MLCA model have been generated with covariates. Conditions varied at different levels in terms of six study factors: 1) the number of latent class indicators, 2) the quality of latent class indicators (i.e., CRPs), 3) the covariate effect at the lower level, 4) the covariate effect at the higher level, 5) the number of groups/higher-level units, and 6) the size of groups/higher-level units. The values specified and varied in this dissertation follow scenarios that are similar to those investigated in related literature, as will be seen in the real data analyses in Chapter 6. Instead of limiting to a few factors with more levels, this dissertation aims to investigate more study factors as how these dimensions of a NP-MLCA model affect the classification accuracy and parameter estimation.  4.1.1 Data Generation In the current dissertation, the statistical software package Mplus (Muthén & Muthén, 1998-2015) were used to conduct the simulation studies. All generated data included 100 replications. For Monte Carlo simulation studies, Mplus has the capability to generate dataset from the researchers™ specified population parameter, and estimate the researcher-specified model using the generated datasets. A sample Mplus syntax file can be found in the Appendix A. In the current dissertation, the following aspects of the model are fixed.  First, the number of higher-level latent classes is set as two, ~ (0.5,0.5). That is, two mixture components at the higher level were generated. The higher-level latent classes are set to be evenly distributed. Half of the groups are from higher-level latent class 1 and the other 23  half are from higher-level latent class 2. It is assumed that higher level units in the same mixture component at the higher level share the same set of parameters.  Second, the number of lower-level latent classes is fixed at three for each group; therefore, two random intercepts of latent class probabilities (i.e., a pair of group-varying random components) were generated for each group. For lower-level latent classes, class 3 is as the reference class. Thus, ===0. Based on the real dataset, the means of two random intercept variables are =0.0610.027, and the residual variance covariance matrix ~0.0610.027,0.727000.435 .  The random intercepts are continuous variables that vary across groups, and in this current dissertation, the means of these random intercepts are allowed to vary across the higher-level latent classes. These random intercepts are used to represent groups™ influences on individuals™ probabilities of being particular lower-level latent classes. Third, using higher-level class 2 as the reference group, the relationship between being in the lower-level unit and the higher-level unit is specified as follows: =1=1)=0.75,=2=1=0.5.   Fourth, higher-level and lower-level covariates are dichotomized from a Normal (0, 1) standard normal distributed variable. They were both dichotomized from zero.  In this dissertation, six study factors were varied in terms of the following conditions:  First, the number of latent class indicators was varied in two conditions: 6 and 12. Second, the quality of latent class indicators (i.e., CRPs) were varied in three conditions: {0.8, 0.2}, {0.7, 
0.3}, and {0.6, 0.4}. Individuals with low CRPs for all of the indicators are in Class 1. Individuals with high CRPs for half of the indicators, and low CRPs for the other half of the 24  indicators, are in Class 2. Individuals with high CRPs for all of the indicators are in Class 3. For example, if the CRP is set at 0.8, individuals in Class 1 will have a 0.8 probability of choosing response 1 on all indicators; individuals in Class 2 have a 0.8 probability of choosing response 1 on half of the indicators, and a 0.2 probability of choosing response 1 on the other half; and individuals in Class 3 have a 0.2 probability of choosing response 1 on all of the items. Third, 
The high-level covariate effect (i.e., regression coefficient) were varied in three conditions: ()=1,1.33,1.5,and 1.75. For each condition, it is fixed as the same across two higher-level mixture components. The effect of the lower-level covariate on latent class 1 as compared with latent class 3 varied in three conditions: ()= 1,1.33,1.5,and 1.75. Fourth, likewise, the lower-level covariate effect on latent class 2 as compared with latent class 3 (i.e., the reference class) varied in three conditions: ()=1,2,3,and 5. Fifth, the number of groups was varied as =50  150. Sixth, the within-group sample size was varied as =30,and 60.  For each of the conditions above, parameterization is carefully calculated, and 100 replication datasets were simulated.  4.2 Evaluation Criteria The performance of the two primary outcome measures was examined. Specifically, classification accuracy was measured in terms of consistent proportion between the true latent class membership and the predicted latent class membership at different levels. The parameter recovery was evaluated in terms of bias, standard error bias and the 95% confidence interval coverage rate of the estimate.  25  4.2.1 Classification Accuracy  In this dissertation, both lower-level classification accuracy and higher-level classification accuracy were evaluated. At each level, the true latent class membership and the predicted class membership from each replication dataset of each condition were extracted. The proportion is the rate of identifying the correct class membership, utilized as the classification accuracy in this dissertation. It was examined and computed separately for the latent classes at the different level.  4.2.2 Bias  For each condition, the bias for each of the following model parameters was evaluated: CRPs and the covariate effects at each level. The bias is defined as the difference between the average estimate and the population parameter, where the average estimate is computed by averaging the parameter estimate over all 100 replications.   =     The Mplus program is used throughout this dissertation.  The logit scale is used for parameterization. Therefore, the bias of an estimate for this parameterization is the difference between the average of the estimate and the population parameter in the logit scale. 4.2.3 Standard Error Bias  Standard error bias is defined by the percentage difference between the population standard error and the average of the estimated standard errors of each parameter estimate over 
all replications. The formula can be expressed as follows:   Standard Error Bias=S.E.AverageStandard Deviation    26   In the best scenario, the average of the estimated standard error should be very close to population standard error. It is noted in Muthén & Muthén (2002) that standard error bias should be smaller than 10% for the parameters in the model.   4.2.4 Coverage Rate  The coverage rate in this dissertation was set at 95%. It was utilized to evaluate the proportion of replication in each parameter estimate contains the population parameter value (Muthén & Muthén, 1998-2015). In Muthén & Muthén (2002), it is recommended that the coverage rate should be at least .91. That is, at least 91 % of replications having true parameter values within the 95% confidence interval.  4.3 Six-way ANOVA  For each outcome measures and evaluation criteria, 6-way ANOVA was applied to examine the effect of the study factors to further understand the importance of different study factors. Effect size  was utilized to compare effects for the practical importance measure. According to Cohen, (1988), a  >0.138  is regarded as a large effect, and a 0.138>>0.059 is corresponded to a medium effect.  is calculated using the following formula:  =   where  is the sum of square of the effect of interest. In this dissertation, it can be main effect, two-way interaction effect, three-way interaction effect, four-way interaction effect and five-way interaction effect in the analyses. From the formula, it is shown that  is the proportion of the total variability of the outcome measure attributable to a factor of interest. When the degree of freedom of the study factor is one, the square root of  is correlation, and 27  when the degree of freedom is larger than 1,  is equivalent to . With its additive feature, the total variance explained by study important factors can be calculated. In this dissertation, strong effects are the focus in Chapter 5, if there are no strong effects in the analysis results, medium effects are extracted and presented.    28  Chapter 5   Simulation Results  The dissertation was designed to evaluate the performance of NP-MLCA in terms of classification accuracy and parameter estimation under 384 conditions.  These conditions are the combination from the following different levels of six study factors:  1) The quality of latent class indicators: 0.6, 0.7, and 0.8, 2) The number of latent class indicators: 6 and 12,   3) The number of groups: 50 and 150,    4) The group size: 30 and 60,  5) The covariate effects at the individual level: (1, 1), (1.33, 2), (1.5, 3), and (1.75, 5), and  6) The covariate effects at the group level: 1, 1.33, 1.5, and 1.75 This chapter contains two parts. The first part presented the results of classification accuracy. The second part presented the results of parameter estimation. In the first part, outcome measures classification accuracy results were organized by different levels. In the second part, the performance of the parameter estimates recovery was discussed by four evaluation criteria: bias, percent standard error bias, mean square error, and confidence interval coverage rate. In each section of evaluation criteria, six parameter estimates in the model were presented: 1) level-1 covariate effect on latent class 1, 2) level-1 covariate effect on latent class 2, 3) level-2 covariate effect, 4) conditional response probabilities in latent class 1, 5) conditional response probabilities in latent class 2, and 6) conditional response probability in latent class 3. In each section of classification accuracy and each parameter estimates evaluation, two-way 29  interaction plots were presented, followed by the results of the six-way analysis of variance. Tables for results of 384 conditions can be found in Appendix B. In the tables, values within acceptable range are marked yellow. Values with abnormal spikes (peaks) are marked red. Combining the number of groups and the group size condition, ten two-way interaction plots are generated in each section of parameter estimates and classification accuracy as descriptive presentation and exploration of the 384 results. They are grouped in the following order:  1) Three two-way interaction plots between the quality of latent class indicators, the number of latent class indicators, and the overall sample size. a. Quality of latent class indicators × Overall samples size b. Number of latent class indicators × Overall sample size c. Quality of latent class indicators × Number of Latent class indicators  2) Three two-way interaction plots of the level-1 covariate effects with the quality of latent class indicators, the number of latent class indicators, and the overall sample size.  a. Level-1 covariate effect × Quality of latent class indicators  b. Level-1 covariate effect × Number of latent class indicators c. Level-1 covariate effect × Overall sample size   3) Three two way interaction plots of the level-2 covariate effects with the quality of latent class indicators, the number of latent class indicators, and the overall 
sample size.  a. Level-2 covariate effect × Quality of latent class indicators  b. Level-2 covariate effect × Number of latent class indicators 30  c. Level-2 covariate effect × Overall sample size   4) One two-way interaction between level-1 covariate effect and level-2 covariate effect.  For each plot presented, the y-axis presented the aggregated evaluation criteria (i.e., bias, standard error bias, and 95% confidence interval coverage rate) across other conditions. For example, plot 1) a is the barplot of biases of the parameter of estimate by quality of indicators and by overall sample sizes, the y axis of this barplot is the averaged bias across other three study factors: number of indicators, level-1 covariate effect, and level-2 covariate effects.  After the ten two-way interaction plots, the six-way ANOVA was conducted. Due to the fact that the analysis includes six main effect, 15 two-way interaction effects, 20 three-way interaction effects, 15 four-way interaction effects, and six five-way interaction effects, in the results, p values for the F test statistics and effect size  are utilized. Among those p-values smaller than .05, strong effects (effect size >0.138 ) are first presented. If there is no strong effects in the results, medium effect sizes (0.059 <  < 0.138) are presented. If there is no statistical significant effect in the analysis, or there is statistical significant effect but only small or negligible effect sizes are identified, the ANOVA result is skipped.  For the sake of the simplicity and interpretability, the estimates of the outcome measures are all in logit scale, while the scales of study factors are transformed for an easier presentation in the following sections. For example, quality of indicators is on the probability scale, and covariate effects are on the odds ratio scale. All the study factors were referred to as the categorical variables in the six-way ANOVA.  31  The results below begin with a summary of the results presented in two sections: the results of classification accuracy and the quality of parameter estimation.  5.1 Summary of the Results Table 1 (below) shows the important effect sizes identified from individual six-way ANOVA results. In the table, the columns represent the main effect or interaction effects, and each row (outcome measure) was examined by one six-way ANOVA and are presented individually the in sections afterwards.  Generally, the main effects and the two-way interactions of the study factors represented in the table are strong effects. Medium effects were only shown when the outcome measures were biases of the level-2 covariate effect, the standard error biases of level-1 covariate effects on latent class 1, and the standard error biases of latent class 1conditoinal response probability. When the outcome measures are biases of level-1 covariate effects on latent class 1, biases and standard error biases of level-1 covariate effects on latent class 2, and standard error biases of latent class 2 and latent class 3 ‚s CRP, neither a strong effect nor medium effect were found for these outcome measures™ differences.   The number of indicators is an important factor in terms of classification accuracy at both levels; 95% CI coverage rates of level-1 covariate effects, and biases and 95% CI coverage rate of CRPs of latent class 1 and latent class 2.   The quality of indicators is an important factor in terms of level-1 classification accuracy; 95% CI coverage rates of level-1 covariate effects on latent class 2 and level-2 covariate effects, biases and 95% CI coverage rates of CRPs of latent class 1 and 3, and 95% CI coverage rates of CRP of latent class 2.  32   Level-1 covariate effect is an important factor in terms of 95% CI coverage rates of both sets of level-1 covariate effects.   The interaction effect of number of indicators and quality of indicators is important for higher-level classification accuracy and biases of CRP of latent class 1. The interaction effect of number of groups and quality of indicators is important for 95% CI coverage rate of level-2 covariate effects.   Biases of level-2 covariate effects, standard error biases of level-1 covariate effects on latent class 2, and standard error biases of CRP of latent class 1, are affected by the interactions between the level-1 covariate effects and the level-2 covariate effects.  In addition to these two study factors, the differences in these outcome measures were also altered by quality of indicators, number of indicators, group size and number of groups. Specifically, group size, number of groups, and quality of indicators affects biases of level-2 covariate effects. Number of indicators and quality of indicators has an effect on standard error biases of latent class 1 CRP, in 
addition to level-1 and level-2 covariate effects. Number of groups, number of indicators, and quality of indicators affects the standard error biases of level-1 covariate effect on latent class 1.    33   Table 1. Summary of Important Effect Sizes from Six-way ANOVA Results Outcome Measures Ni Qi L1Cov Ni * Qi Ng * Qi Ni * Qi                     *L1Cov *L2Cov Gs * Qi      *L1cov *L2cov Ng * Gs *Qi  *L1cov    *L2cov Ng * Ni *Qi *L1cov 
*L2cov Classification Accuracy CW.CR 0.15 0.73 - - - - - - - CB.CR 0.55 - - 0.16 - - - - - Quality of Parameter Estimation  Bias L1Cov.C1 - - - - - - - - - L1Cov.C2 - - - - - - - - - L2Cov - - - - - - 0.07 0.08 - CRP.LC1 0.22 0.19 - 0.24 - - - - - CRP.LC2 0.43 - - - - - - - - CRP.LC3 - 0.16 - - - - - - - Standard Error Bias L1Cov.C1 - - - - - - - - 0.06 L1Cov.C2 - - - - - - - - - L2Cov - - - - - - - - - CRP.LC1 - - - - - 0.06 - - - CRP.LC2 - - - - - - - - - CRP.LC3 - - - - - - - - - 95%CI Coverage Rate L1Cov.C1 0.25 - 0.33 - - - - - - L1Cov.C2 0.19 0.21 0.38 - - - - - - L2Cov - 0.43 - - 0.17 - - - - CRP.LC1 0.24 0.33 - - - - - - - CRP.LC2 0.48 0.28 - - - - - - - CRP.LC3 - 0.73 - - - - - - - Note: CW.CR: Level-1 classification accuracy; CB.CR: Level-2 classification accuracy; L2Cov: Level-2 covariate effect; L1Cov.C1: Level-1 covariate effect on latent class 1; L1Cov.C2: Level-1 Covariate effect on latent class 2; CRP.LC1: Latent class 1 conditional response probability; CRP.LC2: Latent class 2 conditional response probability; CRP.LC3: Latent class 3 conditional response probability; Ni: Number of indicators; Qi: Quality of indicators; Gs: Group size; Ng; Number of groups 34  5.2 Classification Accuracy 5.2.1 Individual-Level Classification Accuracy  Figure 1 to Figure 4 presented the mean individual-level classification accuracy using ten interaction plots. As shown in the plots, when the quality of indicators and the number of indicators increased, the individual-level classification accuracy improved. But the increase of sample size, level-1 covariate effect or the level-2 covariate effect did not show large influence on improving individual-level classification accuracy.    Figure 1. Two-way interaction of study factors on level-1 classification accuracy  a) quality of latent class indicators × overall sample size, b) number of indicators × overall sample size, and c) quality of indicators × number of indicators.  35   Figure 2. Two-way interaction of study factors on level-1 classification accuracy  a) quality of latent class indicators × overall sample size, b) number of indicators × overall sample size, and c) quality of indicators × number of indicators.  Figure 3. Two-way interaction of study factors on level-1 classification accuracy  a) level-2 covariate effect × quality of indicators, b) level-2 covariate effect × number of indicators, and c) level-2 covariate effect × overall sample size. 36   Figure 4. Two-way interaction of study factors on level-1 classification accuracy: level-2 covariate effect × level-1 covariate effects.  Table 2. Study Factors with Strong Effect Sizes on Level-1 Classification Accuracy  Df Sum Sq Mean Sq F value Pr(>F) Effect Size Quality of Indicators 2 6.21 3.10 32801.17 0.000 0.73 Number of Indicators 1 1.26 1.26 13311.03 0.000 0.15  A factorial ANOVA was conducted to determine if the mean level-1 classification accuracy differed based on the six study factors. From Table 2, it is shown that effect sizes are large for quality of indicators (  =0.73) and number of indicators (   =0.15). None of the interaction effects showed large effect sizes. Quality of indicators and number of indicators explain in total of 88% variability of level-1 classification accuracy.  37  5.2.2 Group-Level Classification Accuracy The mean of group-level classification accuracy showed little variation across all study factors and in the two-way interaction plots from Figure 5 to Figure 8. Except for the number of indicators, the differences in the level-2 classification accuracy among different conditions of number of groups, group sizes, covariate effects at two levels, and quality of indicators appeared to be trivial. For number of indicators, the group level classification accuracy is slightly higher for conditions with 6 indicators instead of 12 indicators.  Figure 5. Two-way interaction of study factors on leve-2 classification accuracy a) quality of latent class indicators × overall sample size, b) number of indicators × overall sample size, and c) quality of indicators × number of indicators. 38   Figure 6. Two-way interaction of study factors on level-2 classification accuracy a) level-1 covariate effects × quality of indicators , b) level-1 covariate effects × number of indicators, and c) level-1 covariate effect × overall sample size.  Figure 7. Two-way interaction of study factors on level-2 classification accuracy  a) level-2 covariate effect × quality of indicators, b) level-2 covariate effect × number of indicators, and c) level-2 covariate effect × overall sample size. 39   Figure 8. Two Œway interaction of study factors on level-2 classification accuracy: level-2 covariate effect × level-1 covariate effects. Table 3. Study Factors with Strong Effect Sizes on Level-2 Classification Accuracy  Df Sum Sq Mean Sq F value Pr(>F) Effect Size  Number of Indicators 1 0.12 0.12 4068.73 0.000 0.55 Number of Indicators × Quality of Indicators 2 0.04 0.02 597.30 0.000 0.16  A factorial ANOVA was conducted to determine if the mean level-2 classification accuracy differed based on the six study factors. From Table 3, it is shown that effect sizes are large for number of indicators (   =0.55) and the interaction effect between number of indicators and quality of indicators (  ×  =0.16). In Figure 5(c), the interaction between quality of indicators and number of indicators can be explained. When the quality of indicators is 0.6, the group-level classification accuracy has little difference between 6 indicators and 12 indicators, while in the conditions where quality of indicators is 0.7 or 0.8, the mean group-level classification accuracy for 6 indicators is better than for 12 indicators. The possible reasons are discussed in Chapter 7. 40  5.3 Parameter Estimation Recovery  To explore parameter estimation recovery quality in these 384 conditions, three evaluation criteria are utilized: bias, standard error bias, and coverage rate. The definition and the formula for each of them were described in Chapter 4. For each criterion, six parameter estimates are examined. As before, ten interaction plots are presented to examine the pattern of the outcome measure under different conditions, followed by a six-way ANOVA. Strong effect sizes are reported, and if strong effect sizes cannot be identified, the medium size factors are reported.  5.3.1 Bias 5.3.1.1 Level-1 Covariate Effect on Latent Class 1 Figure 9 to Figure 12 showed no clear pattern regarding the effect of study factors on the level-1 covariate effect on latent class 1. Here these parameter estimates are 1, 1.33, 1.5, and 1.75. Increasing quality of indicators and number of indicators may help decrease the mean bias of the level-1 covariate effect on latent class 1. However, the magnitude of bias may altered by the covariate effect at different levels.  A factorial ANOVA was conducted. The results indicated no statistically significant effect. In other words, these differences between study factors are not important factors that can help explain the variances of mean bias for level-1 covariate effects.  41   Figure 9. Two-way interaction of study factors on mean bias of level-1 covariate effect on latent class 1  a) quality of latent class indicators × overall sample size, b) number of indicators × overall sample size, and c) quality of indicators × number of indicators.  Figure 10. Two-way interaction of study factors on mean bias of level-1 covariate effect on latent class 1 a) level-1 covariate effects × quality of indicators, b) level-1 covariate effects × number of indicators, and c) level-1 covariate effect × overall sample size. 42   Figure 11. Two-way interaction of study factors on mean bias of level-1 covariate effect on latent class 1 a) level-2 covariate effect × quality of indicators, b) level-2 covariate effect × number of indicators, and c) level-2 covariate effect × overall sample size. . Figure 12. Two-way interaction of study factors on mean bias of level-1 covariate effect on latent class 1: level-2 covariate effect × level-1 covariate effects. 43  5.3.1.2 Level-1 Covariate Effect on Latent Class 2 Level-1 covariate effects on latent class 2 are the parameter estimation evaluated in this section. They are 1, 2, 3, and 5. From Figure 13 to Figure 16, there is a pattern showing that the quality of indicators and the number of indicators has an impact on the mean bias of level-1 covariate effect on latent class 2. A further investigation using factorial ANOVA indicated that quality of indicators and the interaction effect of quality of indicators and number of indicators 
are statistically significant (  =7.07,<.01;    ×  =3.93,<.05). However, both effect sizes are small (   = 0.04,   ×  =0.02).   Figure 13. Two-way interaction of study factors on mean bias of level-1 covariate effect on latent class 2: a) quality of latent class indicators × overall sample size, b) number of indicators × overall sample size, and c) quality of indicators × number of indicators. 44   Figure 14. Two-way interaction of study factors on mean bias of level-1 covariate effect on latent class 2: a) quality of latent class indicators × overall sample size, b) number of indicators × overall sample size, and c) quality of indicators × number of indicators.  Figure 15. Two-way interaction of study factors on mean bias of level-1 covariate effect on latent class 2: a) level-2 covariate effect × quality of indicators, b) level-2 covariate effect × number of indicators, and c) level-2 covariate effect × overall sample size. 45   Figure 16. Two-way interaction of study factors on mean bias of level-1 covariate effect on latent class 2: level-2 covariate effects × level-1 covariate effects. 5.3.1.3 Level-2 Covariate Effect The conditions of level-2 covariate effect varied at four levels: 1, 1.33, 1.5, and 1.75, in odds ratio scale. It is shown that under the condition when the quality of indicators is low, the mean bias is relatively large, and when there is no level-2 covariate effect (odds ratio = 1), the mean bias is also relatively large. However, the magnitude of bias is varied by other study factors as well. The pattern for level-2 covariate effect estimate recovery is not very clear under different study factors in Figure 17 to Figure 20. 46   Figure 17. Two-way interaction of study factors on mean bias of level-2 covariate effect: a) quality of latent class indicators × overall sample size, b) number of indicators × overall sample size, and c) quality of indicators × number of indicators.  Figure 18. Two-way interaction of study factors on mean bias of level-2 covariate effect a) level-1 covariate effects × quality of indicators , b) level-1 covariate effects × number of indicators, and c) level-1 covariate effect × overall sample size. 47   Figure 19. Two-way interaction of study factors on mean bias of level-2 covariate effect a) level-2 covariate effect × quality of indicators, b) level-2 covariate effect × number of indicators, and c) level-2 covariate effect × overall sample size.  Figure 20. Two-way interaction of study factors on mean bias of level-2 covariate effect: level-2 covariate effect × level-1 covariate effects.  48  A six-way AVNOA was conducted, there is no strong effect size identified in the result (See Table 4). There are two medium effects: the five-way interaction (number of groups × group size×quality of indicators× Level-1 covariate effects ×Level-2 covariate effects) and the four-way interaction effects (group size×quality of indicators× Level-1 covariate effects ×Level-2 covariate effects). The results indicated that the variance of mean bias estimate of level-2 covariate effect is a function of number of groups, group size, quality of indicators, level-1 covariate effects, and level-2 covariate effects.  Table 4. Study Factors with Medium Effect Sizes on Biases of Level-2 Covariate Effects  Df Sum Sq Mean Sq F value Pr(>F) Effect Size   Ng : Gs : Qi : L1cov : L2cov  18 382258.94 21236.61 1.07 0.443 0.08  Gs : Qi : L1cov : L2cov             18 344690.82 19149.49 0.97 0.529 0.07 Note: Ng: Number of groups; Gs: Group size; Qi: Quality of indicators; L1cov: Level-1 covariate effects; L2cov: Level-2 covariate effects 5.3.1.4 Conditional Response Probability of Latent Class 1  Figure 21 to Figure 24 showed that the mean bias of conditional response probabilities of latent class 1 is trivial, showing good recovery of the estimate. Figure 22 and Figure 23 showed similar pattern when the level-1 covariate effect or level-2 covariate effect increased. Figure 24 did not show clear interaction between level-1 covariate effect and level-2 covariate effect, either. When sample size is 9,000, number of indicators is 12, and level-1 covariate effects are (1,1), (1.33,2), and (1.5, 3), the bias of conditional response probability of latent class 1 can dropped to less than 0.01 (See Appendix Table 24).   49   Figure 21. Two-way interaction of study factors on mean bias of conditional response probabilities of latent class 1: a) quality of latent class indicators × overall sample size, b) number of indicators × overall sample size, and c) quality of indicators × number of indicators.  Figure 22. Two-way interaction of study factors on mean bias of conditional response probabilities of latent class 1: a) level-1 covariate effects × quality of indicators, b) level-1 covariate effects × number of indicators, and c) level-1 covariate effect × overall sample size. 50   Figure 23. Two- way interaction of study factors on mean bias of conditional response probabilities of latent class 1: a) level-2 covariate effect × quality of indicators, b) level-2 covariate effect × number of indicators, and c) level-2 covariate effect × overall sample size.  Figure 24. Two-way interaction of study factors on mean bias of conditional response probabilities of latent class 1: level-2 covariate effects × level-1 covariate effects. 51  A six-way ANOVA was conducted. Table 5 showed three strong effects on mean bias of conditional response probabilities of latent class 1: the interaction effect between number of indicators and quality of indicators, and two main effect: number of indicators quality of indicators. The interaction plot Figure 21 (c) can be referred to interpret the interaction effect. As 
the quality of indicator is 0.6, the mean bias of the estimate for 12 indicators condition is higher 
than that of the 6 indicators condition. While under the condition when quality of indicators is 
0.7 and higher, the mean bias of the estimate for 12 indicators is lower than that of the 6 indicators condition.  Table 5. Study Factors with Strong Effect Sizes on Biases of Conditional Response Probabilities of Latent Class 1  Df Sum Sq Mean Sq F value Pr(>F) Effect Size  Number of Indicators × Quality of Indicators                                         2 2.18 1.09 518.72 0.000 0.24  Number of Indicators                                                     1 1.94 1.94 923.44 0.000 0.22  Quality of Indicators                                                    2 1.68 0.84 399.85 0.000 0.19  5.3.1.5 Conditional Response Probability of Latent Class 2 Figure 25 to Figure 28 showed that the mean bias of conditional response probabilities of latent class 2 is also trivial, showing good recovery of the estimate. Figure 26 and Figure 27 showed similar pattern when the level-1 covariate effect or level-2 covariate effect increased. Figure 28 did not show clear interaction between level-1 covariate effect and level-2 covariate effect, either. Similar to the mean bias of latent class 1, CRPs, when sample size is 9,000, number of indicators is 12, and level-1 covariate effects are (1,1), (1.33,2), and (1.5, 3), the bias of latent class 2 CRPs can dropped to less than 0.01 (See Appendix Table 25).  52   Figure 25. Two-way interaction of study factors on mean bias of conditional response probabilities of latent class 2: a) quality of latent class indicators × overall sample size, b) number of indicators × overall sample size, and c) quality of indicators × number of indicators.  Figure 26. Two-way interaction of study factors on mean bias of conditional response probabilities of latent class 2: a) level-1 covariate effects × quality of indicators , b) level-1 covariate effects × number of indicators, and c) level-1 covariate effect × overall sample size. 53   Figure 27. Two-way interaction of study factors on mean bias of conditional response probabilities of latent class 2: a) level-2 covariate effect × quality of indicators, b) level-2 covariate effect × number of indicators, and c) level-2 covariate effect × overall sample size.  Figure 28. Two-way interaction of study factors on mean bias of conditional response probabilities of latent class 2: level-2 covariate effects × level-1 covariate effects.  54   A six-way ANOVA was conducted to investigate important effect for explaining the variance of the mean bias of latent class 2 CRPs. The results in Table 6 showed that number of indicators has the strong effect (=0.43) on explaining the variance of the biases. Since the degree of freedom of number of indicators is 1, the square root of  indicated the correlation between the factor and the outcome measure. Here, the correlation between number of indicators and the mean bias of latent class 2 CRPs is 0.43=0.66. Also, from the interaction plots Figure 25 (b) (c), Figure 26 (b), and Figure 27 (b), the pattern that the mean bias of condition 12 indicators is consistently smaller than that of condition 6 indicators.  Table 6. Study Factors with Strong Effect Sizes on Biases of Conditional Response Probabilities of Latent Class 2  Df Sum Sq Mean Sq F value Pr(>F) Effect Size  Number of Indicators                                                     1 1.92 1.92 960.00 0.000 0.43  5.3.1.6 Conditional Response Probability of Latent Class 3 Figure 29 to Figure 32 showed that the mean bias of conditional response probabilities of latent class 3 is trivial as well, showing again good recovery of the estimate. Figure 30 and Figure 31 showed similar pattern when the level-1 covariate effect or level-2 covariate effect increased. Figure 29 did not show clear interaction between level-1 covariate effect and level-2 covariate effect, either. Similar to the mean bias of latent class 1 CRPs and latent class 2 CRPs, when sample size is 9,000, number of indicators is 12, and level-1 covariate effects are (1,1), (1.33,2), and (1.5, 3), the bias of latent class 3 CRPs can dropped to less than 0.01 (See Appendix Table 26).  55   Figure 29. Two-way interaction of study factors on mean bias of conditional response probabilities of latent class 3: a) quality of latent class indicators × overall sample size, b) number of indicators × overall sample size, and c) quality of indicators × number of indicators.  Figure 30. Two-way interaction of study factors on mean bias of level-2 covariate effect a) level-1 covariate effects × quality of indicators , b) level-1 covariate effects × number of indicators, and c) level-1 covariate effect × overall sample size. 56   Figure 31. Two-way interaction of study factors on mean bias of conditional response probabilities of latent class 3: a) level-2 covariate effect × quality of indicators, b) level-2 covariate effect × number of indicators, and c) level-2 covariate effect × overall sample size.  Figure 32. Two-way interaction of study factors on mean bias of conditional response probabilities of latent class 3: level-2 covariate effects × level-1 covariate effects.  57  A six-way ANOVA was conducted to investigate important effect for explaining the variance of the mean bias of latent class 3 CRPs. The results in Table 7 showed that the only factor that has strong effect is the quality of indicators (=0.16). In other words, the quality of indicators can explain 16% of the variance of the latent class 3 CRPs. Also, from Figure 29 (a), Figure 30 (a), and Figure 31 (a), we can find consistent patterns that the increase of the quality of indicators decreases the mean bias of the latent class CRPs.  Table 7. Study Factors with Strong Effect Sizes on Biases of Conditional Response Probabilities of Latent Class 3  Df Sum Sq Mean Sq F value Pr(>F) Effect Size  Quality of Indicators                                                    2 0.89 0.44 130.68 0.000 0.16  5.3.2 The Standard Error Estimate Bias 5.3.2.1 Level-1 Covariate Effect on the Latent Class 1 There is no clear pattern shown in Figure 33 to Figure 36. The value in Table 27 in the Appendix is marked in yellow as the absolute value of standard error bias is less than 10%. According to Muthén and Muthén (2002), these values showed acceptable quality of the parameter recovery. They were mostly observed in those conditions with quality of indicators 0.8, and number of indicators 12. When level-1 covariate effects and the level-2 covariate effects are large, the standard error bias of level-1 covariate effect on the latent class 1 tends to increase. But the magnitude varies by other factors as well. Some spikes were observed when quality of indicators is 0.6 (highlighted as red in Table 27 in the Appendix), indicating the instability of the parameter estimate. 58   Figure 33. Two-way interaction of study factors on mean standard error estimate bias of level-1 covariate effects on the latent class 1: a) quality of latent class indicators × overall sample size, b) number of indicators × overall sample size, and c) quality of indicators × number of indicators.  Figure 34. Two-way interaction of study factors on mean standard error estimate bias of level-1 covariate effects on the latent class 1 a) level-1 covariate effects × quality of indicators, b) level-1 covariate effects × number of indicators, and c) level-1 covariate effect × overall sample size. 59   Figure 35. Two-way interaction of study factors on mean standard error estimate bias of level-1 covariate effects on the latent class 1: a) level-2 covariate effect × quality of indicators, b) level-2 covariate effect × number of indicators, and c) level-2 covariate effect × overall sample size.  Figure 36. Two-way interaction of study factors on mean standard error estimate bias of level-1 covariate effects on the latent class 1: level-2 covariate effects × level-1 covariate effects.  60  A six-way ANOVA was conducted. There is no strong effect factor identified. The results in Table 8 showed that the factor with the medium effect in this analysis is a five-way interaction effect: number of groups×number of indicators × quality of indicators × level-1 covariate effects × level-2 covariate effects. In other words, the variance of the mean standard error bias of level-1 covariate effects on latent class 1 is function of these five factors.  Table 8. Study Factors with Medium Effect Sizes on Standard Error Biases of Level-1 Covariate Effects on Latent Class 1  Df Sum Sq Mean Sq F value Pr(>F) Effect Size   Ng : Ni : Qi : L1cov : L2cov  18 409.19 22.73 1.57 0.175 0.06 Note: Ng: Number of groups; Ni: Number of indicators; Qi: Quality of indicators; L1cov: Level-1 covariate effects; L2cov: Level-2 covariate effects 5.3.2.2 Level-1 Covariate Effects on the Latent Class 2 Two-way interaction plots in Figure 37 to Figure 40 showed some spikes in terms of the mean standard error bias for level-1 covariate effects on the latent class 2. They are marked in red in Table 28 in Appendix. Though the red ones are not as many as in the previous section, they still showed the instability of the parameter estimation recovery, especially in conditions when quality of indicators is 0.6. When number of indicators is 12, the quality of indicators is 0.8, and the sample size is large, the mean standard error bias of level-1 covariate effects on the latent class 2 showed general acceptable when the level-1 covariates are (1,1), (1.33, 2) and (1.5, 3) (See Table 28 in Appendix).  A six-way ANOVA was conducted. However, there is no factors with strong or medium effects that can explain the variance of the mean standard error bias for level-1 covariate effects on the latent class 2.  61   Figure 37. Two-way interaction of study factors on mean standard error estimate bias of level-1 covariate effects on the latent class 2: a) quality of latent class indicators × overall sample size, b) number of indicators × overall sample size, and c) quality of indicators × number of indicators.  62   Figure 38. Two-way interaction of study factors on mean standard error estimate bias of level-1 covariate effects on the latent class 2: a) level-1 covariate effects × quality of indicators, b) level-1 covariate effects × number of indicators, and c) level-1 covariate effect × overall sample size. 63   Figure 39. Two-way interaction of study factors on mean standard error estimate bias of level-1 covariate effects on the latent class 2: a) level-2 covariate effect × quality of indicators, b) level-2 covariate effect × number of indicators, and c) level-2 covariate effect × overall sample size.   Figure 40. Two-way interaction of study factors on mean standard error estimate bias of level-1 covariate effects on the latent class 2: level-2 covariate effects × level-1 covariate effects. 64  5.3.2.3 Level-2 Covariate Effect Two-way interaction plots in Figure 41 to Figure 44 showed some spikes in terms of the mean standard error bias for level-2 covariate effect estimate. They are marked in red in Table 29 in Appendix. Though the red one is not as large as the one in the previous sections for level-1 covariate effects, they still showed the instability of the parameter estimation recovery. One thing can be noted is that the column quality of indicators is 0.6, the mean standard error bias is around one time more than the population standard error, even when the number of indicators is 12, and the sample size is second to the largest ( N = 4,500; number of groups = 150, group size 
= 30). However, the best scenario in terms of samples size and quality of indicators, the 
estimation recovery is relatively the best where the sample size is 4500 (group size = 30), and 
quality of indicators is 0.8. Comparing these standard error bias with the column where the 
quality of indicators is the same, 0.8, and the group size is larger (group size = 60), the standard error bias increased rather decreased, no matter the magnitude difference in level-1 covariate effects and level-2 covariate effects (See Table 29 in Appendix).  A six-way ANOVA was conducted. However, there is no factors having strong or medium effects that can explain the variance of the standard error bias for level-2 covariate effects.  65   Figure 41. Two-way interaction of study factors on mean standard error estimate bias of level-2 covariate effect: a) quality of latent class indicators × overall sample size, b) number of indicators × overall sample size, and c) quality of indicators × number of indicators.  Figure 42. Two-way interaction of study factors on mean standard error estimate bias of level-2 covariate effect: a) level-1 covariate effects × quality of indicators , b) level-1 covariate effects × number of indicators, and c) level-1 covariate effect × overall sample size. 66   Figure 43. Two-way interaction of study factors on mean standard error estimate bias of level-2 covariate effect: a) level-2 covariate effect × quality of indicators, b) level-2 covariate effect × number of indicators, and c) level-2 covariate effect × overall sample size.  Figure 44. Two-way interaction of study factors on mean standard error estimate bias of level-2 covariate effect: level-2 covariate effects × level-1 covariate effects.  67  5.3.2.4 Conditional Response Probability of Latent Class 1  It is shown in the figures that when sample size is small, spikes showed. In other words, the standard error recovery is relatively more stable when the number of groups is 150 (Also see Table 30 in Appendix). Except the condition (1.75, 5) for level-1 covariate effect, when number of groups is 150, group size is 60, and number of indicator is 12, the standard error bias is around 0.01 across all levels of level-2 covariate effects, showing good quality of parameter recovery.   Figure 45. Two-way interaction of study factors on mean standard error estimate bias of conditional response probabilities of latent class 1: a) quality of latent class indicators × overall sample size, b) number of indicators × overall sample size, and c) quality of indicators × number of indicators. 68   Figure 46. Two-way interaction of study factors on mean standard error estimate bias of conditional response probabilities of latent class 1: a) level-1 covariate effects × quality of indicators , b) level-1 covariate effects × number of indicators, and c) level-1 covariate effect × overall sample size.  69   Figure 47. Two-way interaction of study factors on mean standard error estimate bias of conditional response probabilities of latent class 1: a) level-2 covariate effect × quality of indicators, b) level-2 covariate effect × number of indicators, and c) level-2 covariate effect × overall sample size. 70   Figure 48. Two-way interaction of study factors on mean standard error estimate bias of conditional response probabilities of latent class 1: level-2 covariate effects × level-1 covariate effects. A six-way ANOVA was conducted, there was no factor showed strong effect. The only one medium effect is the four way interaction among number of indicators, quality of indicators, level-1 covariate effects and level-2 covariate effects. That is, the variance of standard error bias is a function of these four factors.  Table 9. Study Factors with Medium Effect Sizes on Standard Error Biases of Conditional Response Probabilities of Latent Class 1  Df Sum Sq Mean Sq F value Pr(>F) Effect Size   Ni : Qi : L1cov: L2cov            18 1121.60 62.31 0.95 0.546 0.06 Note: Ni: number of indicators; Qi: quality of indicators; L1cov: Level-1 covariate effects; L2cov: Level-2 covariate effects 5.3.2.5 Conditional Response Probability of Latent Class 2 Two-way interaction plots in Figure 49 to Figure 52 does not show clear pattern in term of important study factors. Examining Table 31 in the Appendix, large values in the table still can be identified especially in the condition of quality of indicators 0.6 and 12 latent class 71  indicators. Since latent class 2 is a mixed group with heterogeneous CRP for indicators. It is within expectation that when the number of indicators increase, the model needs to deal with more heterogeneous information from the indicators, resulting in unstable parameter estimation. 
Table 31 suggests when there may be a latent class with mixed pattern of indicators, it is better to have large samples size (N = 9000), using quality of indicators at least 0.8.  A six-way ANOVA was conducted. However, there is no factors with strong or medium effects that can explain the variance of the standard error bias of latent class 2 CRPs.   Figure 49. Two-way interaction of study factors on mean standard error estimate bias of conditional response probabilities of latent class 2: a) quality of latent class indicators × overall sample size, b) number of indicators × overall sample size, and c) quality of indicators × number of indicators. 72   Figure 50. Two-way interaction of study factors on mean standard error estimate bias of conditional response probabilities of latent class 2: a) level-1 covariate effects × quality of indicators, b) level-1 covariate effects × number of indicators, and c) level-1 covariate effect × overall sample size.  73   Figure 51. Two-way interaction of study factors on mean standard error estimate bias of conditional response probabilities of latent class 2: a) level-2 covariate effect × quality of indicators, b) level-2 covariate effect × number of indicators, and c) level-2 covariate effect × overall sample size.  Figure 52. Two-way interaction of study factors on mean standard error estimate bias of conditional response probabilities of latent class 2: level-2 covariate effects × level-1 covariate effects. 74  5.3.2.6 Conditional Response Probability of Latent Class 3 Two-way interaction plots in Figure 53 to Figure 56 does not show clear pattern in term of important study factors, but there are some spikes scattered in these conditions. Examining Table 31 in the Appendix, mostly, large values in the table showed when condition of quality of indicators is 0.6 and the number of latent class indicators is 12. It is also indicated that except for the condition of level-1 covariate effect (1.75, 5), when quality of indicators is 0.8, and the number of groups is 150, the parameter recovery for latent class 3 CRPs is acceptable.  A six-way ANOVA was conducted. However, there is no study factors having strong or medium effects that can explain the variance of the standard error bias of latent class 2 CRPs. In other words, the difference between levels of study factors is small or trivial.   Figure 53. Two-way interaction of study factors on mean standard error estimate bias of conditional response probabilities of latent class 3: a) quality of latent class indicators × overall sample size, b) number of indicators × overall sample size, and c) quality of indicators × number of indicators. 75   Figure 54. Two-way interaction of study factors on mean standard error estimate bias of conditional response probabilities of latent class 3: a) level-1 covariate effects × quality of indicators, b) level-1 covariate effects × number of indicators, and c) level-1 covariate effect × overall sample size.  76   Figure 55. Two-way interaction of study factors on mean standard error estimate bias of conditional response probabilities of latent class 3: a) level-2 covariate effect × quality of indicators, b) level-2 covariate effect × number of indicators, and c) level-2 covariate effect × overall sample size.  Figure 56. Two-way interaction of study factors on mean standard error estimate bias of conditional response probabilities of latent class 3: level-2 covariate effects × level-1 covariate effects. 77  5.3.3 95% Confidence Interval Coverage Rate 5.3.3.1 Level-1 Covariate Effect on the Latent Class 1  Figure 57 to Figure 60 showed no clear interaction between study factors, but consistent pattern in term of single study factors. For example, the average 95% CI coverage rate is higher when number of indicators is 12 rather than 6, quality of indicators 0.8 rather than 0.7 or 0.6. Table 33 for the 95% CI coverage rate is examined for each condition, values equal and larger than 0.91 is marked as yellow, indicating good coverage rate. It is shown that when number of 
indicators is 6, good coverage rate only showed when level-1 covariate effect is (1,1). When number of indicators is 12, 95% CI rate is good when the quality of indicators is 0.8.   Figure 57. Two-way interaction of study factors on mean 95% confidence interval coverage rates of level-1 covariate effects on the latent class 1: a) quality of latent class indicators × overall sample size, b) number of indicators × overall sample size, and c) quality of indicators × number of indicators. 78   Figure 58. Two-way interaction of study factors on mean 95% confidence interval coverage rates of level-1 covariate effects on the latent class 1 a) level-1 covariate effects × quality of indicators, b) level-1 covariate effects × number of indicators, and c) level-1 covariate effect × overall sample size.  79   Figure 59. Two-way interaction of study factors on mean 95% confidence interval coverage rates of level-1 covariate effects on the latent class 1: a) level-2 covariate effect × quality of indicators, b) level-2 covariate effect × number of indicators, and c) level-2 covariate effect × overall sample size.  Figure 60. Two-way interaction of study factors on mean 95% confidence interval coverage rates of level-1 covariate effects on the latent class 1: level-2 covariate effects × level-1 covariate effects. 80   A six-way ANOVA was conducted. Table 10 showed factors with strong effect on 95% CI coverage rate of level-1 covariate effect on latent class 1. The two strong factors are level-1 covariate effects and number of indicators (  =0.33,   =0.25). In total, they can explain 58% of the variance of 95% CI coverage rates of level-1 covariate effect on latent class 1.  Table 10. Study Factors with Strong Effect Sizes on 95% Confidence Interval Coverage Rate of Level-1 Covariate Effects on the Latent Class 1  Df Sum Sq Mean Sq F value Pr(>F) Effect Size  Level-1 Covariate Effects 3 1.64 0.55 1465.52 0.000 0.33 Number of Indicators 1 1.21 1.21 3246.67 0.000 0.25 5.3.3.2 Level-1 Covariate Effect on the Latent Class 2  This section presents the larger set of the level-1 covariate effect. They are 1, 2, 3, and 5. Figure 61 to 64 showed also very consistent pattern that different levels of single study factor have influences on 95% coverage rate of level-1 covariate effect on latent class 2. However, there is no clear interaction pattern shown in these figures. Table 34 in the Appendix also presented similar patterns as they are in Table 33. When the model is under conditions with only 6 indicators, only when level-1 covariate effect is (1,1), are the 95% CI coverage rates equal or larger than 0.91, in the acceptable range. When the conditions are involved 12 indicators , the number of acceptable 95% CI coverage rates increased; however, these happened only when quality of indicators is 0.8, and level-1 covariate effects are (1,1) and (1.33, 2). When level-1 covariate effects are (1.5, 3) and (1.75,5), the 95% CI coverage rate fell out of the acceptable range.  81   Figure 61. Two-way interaction of study factors on mean 95% confidence interval coverage rates of level-1 covariate effects on the latent class 2: a) quality of latent class indicators × overall sample size, b) number of indicators × overall sample size, and c) quality of indicators × number of indicators.  82   Figure 62. Two-way interaction of study factors on mean 95% confidence interval coverage rates of level-1 covariate effects on the latent class 2: a) level-1 covariate effects × quality of indicators, b) level-1 covariate effects × number of indicators, and c) level-1 covariate effect × overall sample size.  83   Figure 63. Two-way interaction of study factors on mean 95% confidence interval coverage rates of level-1 covariate effects on the latent class 2: a) level-2 covariate effect × quality of indicators, b) level-2 covariate effect × number of indicators, and c) level-2 covariate effect × overall sample size.  Figure 64. Two-way interaction of study factors on mean 95% confidence interval coverage rates of level-1 covariate effects on the latent class 2: level-2 covariate effects × level-1 covariate effects. 84   A six-way ANOVA was conducted. Table 11 below showed that strong factor for explaining 95%CI coverage rates variances of level-1 covariate effects on the latent class 2 level-1 covariate effects (  =0.38), quality of indicators (  =0.21), and number of indicators (  =0.19). In total, they can explain 78% variances of 95% CI coverage rates of level-1 covariate effect on the latent class 2. Referred back to the two-way interaction plots above, for 95% CI coverage rates of level-1 covariate effects on the latent class 2, higher effects showed lower 95% CI coverage rates, higher quality indicators has higher coverage rates, and higher number of indicators has higher coverage rates as well. Table 11. Study Factors with Strong Effect Sizes on 95% Confidence Interval Coverage Rate of Level-1 Covariate Effects on the Latent Class 2  Df Sum Sq Mean Sq F value Pr(>F) Effect Size  Level-1 Covariate Effects 3 2.88 0.96 2116.40 0.000 0.38 Quality of Indicators 2 1.61 0.81 1774.95 0.000 0.21 Number of Indicators 1 1.45 1.45 3186.73 0.000 0.19 5.3.3.3 Level-2 Covariate Effect  Similar to the 95% CI coverage rates of level-1 covariate effects, Figure 65 to Figure 68 showed no interaction patterns but main effect patterns. Examining Table 35 for the acceptable individual coverage rates for each condition, quality of indicators 0.8 showed consistently acceptable coverage rate across other factors. When sample size is 9,000 and number of indicators is 12, even under the condition that quality of indicators is 0.6, the 95% CI coverage rate is acceptable. In other words, for the 95% CI coverage rate of level-2 covariate effect, large sample size can compensate for the condition of low quality of indicators. However, it was not 
applied to the condition where level-2 covariate effect is 1.75. The coverage rate was less than 0.91 no matter the sample size is large or the number of indicators is large.  85   Figure 65. Two-way interaction of study factors on mean 95% confidence interval coverage rates of level-2 covariate effects: a) quality of latent class indicators × overall sample size, b) number of indicators × overall sample size, and c) quality of indicators × number of indicators.  Figure 66. Two-way interaction of study factors on mean 95% confidence interval coverage rates of level-2 covariate effects: a) level-1 covariate Effects × quality of indicators, b) level-1 covariate effects × number of indicators, and c) level-1 covariate effect × overall sample size. 86   Figure 67. Two-way interaction of study factors on mean 95% confidence interval coverage rates of level-2 covariate effects: a) level-2 covariate effect × quality of indicators, b) level-2 covariate effect × number of indicators, and c) level-2 covariate effect × overall sample size.  Figure 68. Two-way interaction of study factors on mean 95% confidence interval coverage rates of level-2 covariate effects: level-2 covariate effects × level-1 covariate effects.  87   A six-way ANOVA was conducted. Table 12 below showed two strong effects for 95% CI coverage rate of level-2 covariate effect: quality of indicators, and the interaction effect of number of groups and quality of indicators. This two way interaction effect of number of groups and quality of indicators can be referred to Figure 65(a) when the quality of indicators is 0.6, 
number of groups has an impact on 95% CI coverage rate, conditions with higher number of 
groups has higher 95% CI coverage rate. However, while the quality of indicators is 0.7 or larger, the importance of number of groups has a smaller impact on the changing 95% CI coverage rate.  Table 12. Study Factors with Strong Effect Sizes on 95% Confidence Interval Coverage Rate of Level-2 Covariate Effect   Df Sum Sq Mean Sq F value Pr(>F) Effect Size  Quality of Indicators 2 4.19 2.09 6976.40 0.000 0.43 Number of Groups : Quality of Indicators 2 1.70 0.85 2826.87 0.000 0.17  5.3.3.4 Conditional Response Probability of Latent Class 1   Figure 69 to Figure 72 showed no interaction pattern between study factors on the 95% CI coverage rate of latent class 1 CRP; however, the main effect is quite obvious. Examining Table 36, it is showed that the acceptable 95% CI coverage rate can only be found for the conditions involved 12 indicators and quality of indicators 0.8. When 12 indicators were involved, the increase of level-1 covariate effect lowers the 95% CI coverage rate of latent 1 CRPs. However, the increasing of sample size can compensate for this situation, when the 
number of groups increased to 150, 95%CI coverage rates are acceptable across all ranges of leve-1 covariate effects.   88   Figure 69. Two-way interaction of study factors on mean 95% confidence interval coverage rates of conditional response probabilities of latent class 1: a) quality of latent class indicators × overall sample size, b) number of indicators × overall sample size, and c) quality of indicators × number of indicators.  89   Figure 70. Two-way interaction of study factors on mean 95% confidence interval coverage rates of conditional response probabilities of latent class 1: a) level-1 covariate Effects × quality of indicators, b) level-1 covariate effects × number of indicators, and c) level-1 covariate effect × overall sample size.  90   Figure 71. Two-way interaction of study factors on mean 95% confidence interval coverage rates of conditional response probabilities of latent class 1: a) level-2 covariate effect × quality of indicators, b) level-2 covariate effect × number of indicators, and c) level-2 covariate effect × overall sample size.  Figure 72. Two-way interaction of study factors on mean 95% confidence interval coverage rates of conditional response probabilities of latent class 1: level-2 covariate effects × level-1 covariate effects. 91  However, in the results of six-way ANOVA, Table 13 indicated no interaction effect between number of groups and number of indicators. The two factors with strong effect on 95% CI coverage rates of latent class 1 CRPs are quality of indicators and number of indicators. From 
Table 36 in the Appendix, Figure 70 (a)(b) and Figure 71(a)(b), the plots indicated that higher quality or higher number of indicators has higher 95% CI coverage rates. In total, these two factors can explain 57% variance of latent class 1 CRPs™ 95% CI coverage rates.  Table 13. Study Factors with Strong Effect Sizes on 95% Confidence Interval Coverage Rate of Conditional Response Probability of Latent Class 1  Df Sum Sq Mean Sq F value Pr(>F) Effect Size  Quality of Indicators 2 1.29 0.65 1677.89 0.000 0.33 Number of Indicators 1 0.93 0.93 2416.95 0.000 0.24  5.3.3.5 Conditional Response Probability of Latent Class 2  The same as the 95%CI coverage rate of latent class 1 CRP, 95%CI coverage rate of latent class 2 CRP showed no interaction patterns between study factors in Figure 73 to Figure 77 below. However, the consistent pattern may indicate the effect of single study factor on latent class CRP™s 95% CI coverage rates. A statistical analysis is necessary to conclude whether the differences observed in these figures for single study factors are statistically significant.   Examining Table 37 in the Appendix, it is shown that quality of indicators 0.8 combining with 12 indicators showed generally acceptable 95% CI coverage rates of latent class 2 CRPs. When quality of indicators is 0.7 and 12 indicators involved, acceptable 95% CI coverage rates only showed in conditions with samples size 3,000 and larger.   92   Figure 73. Two-way interaction of study factors on mean 95% confidence interval coverage rates of conditional response probabilities of latent class 2: a) quality of latent class indicators × overall sample size, b) number of indicators × overall sample size, and c) quality of indicators × number of indicators.  93   Figure 74. Two-way interaction of study factors on mean 95% confidence interval coverage rates of conditional response probabilities of latent class 2: a) level-1 covariate effects × quality of indicators, b) level-1 covariate effects × number of indicators, and c) level-1 covariate effect × overall sample size.  94   Figure 75. Two-way interaction of study factors on mean 95% confidence interval coverage rates of conditional response probabilities of latent class 2: a) level-2 covariate effect × quality of indicators, b) level-2 covariate effect × number of indicators, and c) level-2 covariate effect × overall sample size.  Figure 76. Two-way interaction of study factors on mean 95% confidence interval coverage rates of conditional response probabilities of latent class 2: level-2 covariate effects × level-1 covariate effects. 95   A six-way ANOVA was conducted to see if the 95% CI coverage rates differed based on six study factors. Table 14 below showed that two factors have strong effect on 95% CI coverage rates of latent class 2 CRPs: number of indicators (  =.48) and quality of indicators (   =.28). In total, 76% of the variances of 95%CI coverage rate of latent class CRPs were explained by these two factors. Figure 74 (a)(b) and Figure 75 (a)(b) showed that conditions with higher number of indicators or higher quality of indicators have higher 95% CI coverage rate in terms of latent class 2 CRPs.  Table 14. Study Factors with Strong Effect Sizes on 95% Confidence Interval Coverage Rate of Conditional Response Probability of Latent Class 2  Df Sum Sq Mean Sq F value Pr(>F) Effect Size  Number of Indicators 1 1.13 1.13 4817.68 0.000 0.48 Quality of Indicators 2 0.66 0.33 1410.36 0.000 0.28  5.3.3.6 Conditional Response Probability of Latent Class 3  Figure 77 to Figure 80 showed the same pattern as those for latent class 1 CRPs and Latent class 2 CRPs. However, unlike latent class 1 and latent class 2 CRPs, conditions with 12 indicators and quality of indicators 0.7 did not show acceptable 95% coverage rate for CRPs in latent class 3 (as seen in Table 38 in the Appendix). Also, when level-1 covariate effects are high and level-2 covariate effect is 1.75, the results also showed unacceptable 95% CI coverage rate, especially when the sample size is as small as 1,500.  96   Figure 77. Two-way interaction of study factors on mean 95% confidence interval coverage rates of conditional response probabilities of latent class 3: a) quality of latent class indicators × overall sample size, b) number of indicators × overall sample size, and c) quality of indicators × number of indicators.  97   Figure 78. Two-way interaction of study factors on mean 95% confidence interval coverage rates of conditional response probabilities of latent class 3: a) level-1 covariate Effects × quality of indicators, b) level-1 covariate effects × number of indicators, and c) level-1 covariate effect × overall sample size.  98   Figure 79. Two-way interaction of study factors on mean 95% confidence interval coverage rates of conditional response probabilities of latent class 3: a) level-2 covariate effect × quality of indicators, b) level-2 covariate effect × number of indicators, and c) level-2 covariate effect × overall sample size.  Figure 80. Two-way interaction of study factors on mean 95% confidence interval coverage rates of conditional response probabilities of latent class 3: level-2 covariate effects × level-1 covariate effects. 99   A six-way ANOVA was conducted to examine whether the 95% CI coverage rates differed based on six study factors. It is indicated in Table 15 that the only strong effect is quality of indicators (=0.73). In other words, quality of indicators itself can explain 73% variances of 95% CI coverage rates of latent class 3 CPR. It can be concluded that higher quality of indicators results in better 95% CI coverage rates of latent class 3 CRPs.  Table 15. Study Factors with Strong Effect Sizes on 95% Confidence Interval Coverage Rate of Conditional Response Probability of Latent Class 3  Df Sum Sq Mean Sq F value Pr(>F) Effect Size  Quality of Indicators 2 2.81 1.41 3283.66 0.000 0.73    100  Chapter 6   Empirical Study 6.1 Data Collection, Measures, and Covariates Program-evaluation data from the Michigan 21st Century Community Learning Centers Grant Program (21st CCLC) will be utilized to illustrate the model in this dissertation. The program was created in 2002 to provide learning opportunities and activities for children in low-performing schools in Michigan by the state™s Department of Education. It covers academic-enrichment and youth-development activities, drug- and violence prevention programs, and art, music and recreation. Applicants for grants can be school districts, public schools, universities, and nonprofit/community-based organizations. The funded grantees are evaluated yearly, and at the end of the program, teachers are asked to evaluate the students who participated as to 
whether they improved in terms of classroom behavior and homework performance. For purposes of this dissertation, individual student improvement was measured in terms of the following 10 items: 1) Turning in homework on time, 2) Completing homework to your 
[the teacher™s] satisfaction, 3) Participating in class, 4) Volunteering, 5) Attending class regularly, 6) Being attentive in class, 7) Behaving well in class, 8) Academic performance, 9) Coming to 
school motivated to learn, and 10) Getting along well with other students. If students had participated the program for more than 30 days, their teachers would be asked to mark the above 
items on a seven-point scale: significant decline, moderate decline, slight decline, no change, slight improvement, moderate improvement and significant improvement. For the sake of 
balanced sample size, they were dichotomized into improved (slight improvement and above) and unimproved (no change and below). This dissertation™s focus is on students in need of improving their classroom behavior and homework performance. In the case of students who took part in multiple programs, we 101  randomly selected one program in which they participated for analysis. In addition, for estimation purposes, sites with less than five students were removed from the analysis. There are 29 sites removed due to small site sizes, the sample size were down from 6684 to 6580. The 10 latent class indicators and covariates are complete cases. Statistical software R (R Core Team, 2016) was used for data cleaning, and Mplus (Muthén & Muthén, 1998-2015) was utilized for model estimation.  The procedures above resulted in data on 6,580 individuals and 289 sites used for modeling, and the 10 teacher-evaluation items were used as the measured indicators for individual-level classification. The 289 sites at the higher level units were classified using the model specified in equations (23). Because this sample data is intended to illustrate the proposed methodology, only a limited set of variables are conditioned in the model. Covariates at the 
individual level, such as gender, dummy-coded ethnicities, and activities participation hours, were explored. Covariates at the grantee level, such as grantee type, number of non-academic days, and number of program staff, were explored. The distributions of the levels of ten outcome measures before and after dichotomization are shown in Figures 81 and 82. Both of the figures were from 6,580 complete cases. Number in bars in Figure 82 showed the subgroup sizes.       102   Figure 81. The bar plot of ten observed indicators (measures) before dichotomization. 103   Figure 82. The bar plot of ten observed indicators (measures) after dichotomization. 104  6.2 Results of the Study Table 16 and Figure 83 showed the item response probabilities of individuals endorsing the fiimprovedfl answer given their latent class membership in NP-MLCA model and the conditional NP-MLCA model (NP-MLCA model with covariates at two different levels; for example, free lunch status for the individual level, and total number of student in the regular school year). As shown in Table 16, the conditional response probabilities are very close across these two models, indicating that the inclusion of the covariates does not change the conditional 
response probabilities. It is also suggested that the assumption of the independence between covariates and indicators held. Since the item response probabilities of these two models were very close. Only the probability pattern of NP-MLCA is presented in Figure 83.  The main role of the item response probabilities is to characterize the individual latent classes. As shown in the table, individuals in Improved Group endorsed highly (> 0.93) on the fiImprovedfl category. On the contrary, individuals classified in Non-improved Group endorsed low (0.15 and less) on the fiImprovedfl category. The third class, referred to fiMixed Groupfl, endorsed moderately across these ten indicators. However, if we see closely at the probability 
pattern on these ten items in this group, we can find that the individuals™ school behavior were improved, but they struggled more with the academia related work.    
 105  Table 16.  Conditional Response Probabilities from NP-MLCA and Conditional NP-MLCA Models  NP-MLCA  (N=6580, J = 289) Conditional NP-MLCA  (N=6531, J = 288) Conditional Response Probabilities Improved Group Non-improved Group Mixed Group Improved Group Non-improved Group Mixed Group HW on time       Improved 0.928 0.043 0.387 0.935 0.045 0.387 No change / improve 0.072 0.957 0.613 0.065 0.955 0.613 HW satisfaction       Improved 0.942 0.025 0.350 0.949 0.026 0.353 No change / improve 0.058 0.975 0.650 0.051 0.974 0.647 Class Participation       Improved 0.931 0.023 0.338 0.933 0.024 0.348 No change / improve 0.069 0.977 0.662 0.067 0.976 0.652 Volunteering       Improved 0.968 0.124 0.660 0.969 0.127 0.666 No change / improve 0.032 0.876 0.340 0.031 0.873 0.334 Class Attendance       Improved 0.983 0.151 0.730 0.983 0.154 0.734 No change / improve 0.017 0.849 0.270 0.017 0.846 0.266 Attentive in Class       Improved 0.989 0.007 0.525 0.990 0.008 0.537 No change / improve 0.011 0.993 0.475 0.010 0.992 0.463 Class Behavior       Improved 0.979 0.023 0.631 0.979 0.025 0.641 No change / improve 0.021 0.977 0.369 0.021 0.975 0.359 Academic Performance       Improved 0.942 0.005 0.300 0.944 0.005 0.311 No change / improve 0.058 0.995 0.700 0.056 0.995 0.689 Motivated to Learn       Improved 0.992 0.008 0.518 0.993 0.008 0.531 No change / improve 0.008 0.992 0.482 0.007 0.992 0.469 Getting Along with Other Students       Improved 0.974 0.037 0.583 0.975 0.039 0.593 No change / improve 0.026 0.963 0.417 0.025 0.961 0.407    106   Table 17. Model Fit Information and Classification Quality    
  Figure 83. The conditional response probability patterns on ten observed indicators in NP-MLCA model.    00.10.20.30.4
0.50.6
0.70.8
0.91Conditional Response Probabilities Observed Indicators Conditional Response Probabilities of Observed Indicators Improved groupMixed groupNon-improvedgroup NP-MLCA Conditional NP-MLCA Number of estimated parameters 35 51 Entropy 0.870 0.892 AIC 51266.563 50730.029 BIC 51504.276 51076.029 107   Figure 84. The composition of site-level latent classes in NP-MLCA model.   Figure 85. The composition of site-level latent classes in conditional NP-MLCA model. 24.55% 61.41% 31.31% 18.36% 44.14% 20.23% 0%10%20%30%40%50%
60%70%80%90%100%Type I Sites (64.32%)Type II Sites (35.68%)The Composition of the Site-level Latent Classes - Unconditional NP-MLCA model Improved groupMixed groupNon-improved group28.01% 65.94% 29.71% 20.27% 42.28% 13.80% 0%10%20%
30%40%50%60%70%80%90%100%Type I Sites (73.48%)Type II Sites (26.52%)The Composition of the Site-level Latent Classes  - Conditional NP-MLCA Improved groupMixed groupNon-Improved group108   Table 18. The Level-1 Covariate Effects on Level-1 Latent Class Solution (Scale: Odds Ratio) Level Π1 Predictors Reference Group Comparison of non-improved group to improved group Comparison of mixed group to improved group Female Male 0.79** 0.86* White Non-White 0.89 1.25* Special Education No 0.96 1.56* Non-Academic Day  1.01*** 1.01** Note: * p <.05, ** p<.01, *** p < .001  According to the results in Table 18, females are more likely to be in the improved group, as opposed to the non-improved and mixed groups. For females, the odds of being categorized in the improved group are 1.27 (1/0.79) times more likely than the odds for a male categorized in an improved group. The odds of white students to be categorized in a mixed group are 1.25 times more likely than the odds of a non-white student. Special education students are 56% more likely to be in the mixed group rather than in the improved group. For students enrolled one more non-academic day, we would see about 1% increase in the odds of being categorized into the mixed group or the non-improved group. No level-2 covariates were statistically significant in the model.      109  Chapter 7   Summary and Discussion 7.1 Summary and Implication of the Simulation Finding The purpose of this study is to investigate the classification accuracy and the quality of parameter recovery for covariate effect at both levels and CRPs of latent classes in NP-MLCA across six study factors. A total of 384 combinations from six study factors are provided below:  All the discussion and implications in this chapter are restricted to these explored conditions as well.  1) The quality of latent class indicators: 0.6, 0.7, and 0.8, 2) The number of latent class indicators: 6 and 12,   3) The number of groups: 50 and 150, 
4) The group size: 30 and 60,  
5) The covariate effects at the individual level: (1, 1), (1.33, 2), (1.5, 3), and (1.75, 5), and  6) The covariate effects at the group level: 1, 1.33, 1.5, and 1.75  For classification accuracy, the result showed that number of indicators, quality of indicators, or the interaction of both are factors that have a strong effect on improving the classification accuracy. Level-1 covariate effect and level-2 covariate effect does not have impact on classification accuracy at both levels, which, in terms of 1-step procedures, is an exhilarating result. The fact that the 1-step procedure incorporates the covariates at two levels within the estimation procedure means that it could potentially be criticized because different 
incorporated covariates may result in different classification solution. The results provided here 110  however reveal that it does not matter at which level the covariates are included or to what extent the covariate effects are, their effect on classification accuracy is not strong. In other words, they are not strong factors in affecting the classification accuracy. However, this result is restricted to 
the design and assumptions made in this dissertation only. This means that this result does not 
apply to a scenario where there are unmeasured covariates affecting indicators. The result also 
cannot be generalized to a situation where the level-1 covariates and level-2 covariates have a potential interaction effect. The limitations of this dissertation will be fully covered in section 7.3 of this chapter.    Quality of parameter recovery was evaluated based on parameter estimate bias, standard error bias, and 95% CI coverage rate. In this dissertation, there are three latent classes specified at the individual level, resulting in two covariate effects for each incorporated covariate. These two covariate effects were called level-1 covariate effect on latent class 1 and level-1 covariate effect on latent class 2. These were designed in pairs but with different magnitudes, with the design being influenced by the exploration of previous studies. The one on latent class 2 has a stronger effect than the one on latent class 1.  In the results of parameter recovery, none of the factors showed a strong or medium effect in explaining the variability of biases of these two parameter estimates. For the standard error bias, the variability of the standard error bias of level-1 covariate effect on latent class 2 is a function of a five-way interaction among number of groups, number of indicators, quality of indicators, level-1 covariate effects, and level-2 covariate effects. In other words, when the level-1 covariate is regressed to a mixed latent class, the standard error of the estimate can be sensitive to these five factors. Only when standard 95% CI coverage rate of these two effects are the 
outcome of interest, can factors with strong effects be identified. Number of indicators and level-111  1 covariate effects has a strong effect on explaining the variability of 95% CI coverage rate of the level-1 covariate effect on latent class 1. As for the 95% CI coverage rate of level-1 covariate effect on latent class 2, the effects of number of indicators, quality of indicators and level-1 covariate effects reveal themselves to be strong. In total, these three study factors can explain 78% of the total variance of the 95% CI coverage rate of level-1 covariate effect on latent class 2.     For the quality of level-2 covariate effect parameter recovery, the variability of biases is shown to result from the interaction effect of five factors: number of groups, group size, quality of indicators, level-1 covariate effects and level-2 covariate effects. The variability of 95% CI coverage rates are a function of the number of groups and quality of indicators. 43% of the total 
variance is accounted for by the effect of quality of indicators. From this, it can be concluded that large meaningful differences are present between the three levels of quality of indicators.   Though the effect of samples size on outcome evaluation criteria is moderated by other study factors, it is worth noting that differences in sample size has an effect on the variability of the level-2 covariate effect biases and 95% CI coverage rates, and the variability of standard error bias of the level-1 covariate effect on latent class 1. Both number of groups and groups sizes affect the quality of level-2 covariate effect, while only number of groups produces an effect on level-1 covariate effect on latent 1. In addition, the effects of samples size on the variability of these outcome measures are mostly medium, except for the fact that the interaction 
effect of number of groups and quality of indicators on the variability of 95% CI coverage rate is large.      In general, the variability of the bias and 95% CI coverage rates of CRP of latent classes can be mostly accounted for by the effect of the number of indicators, the effect of quality of 112  indicators or the interaction effect of the two together. In other words, large meaningful differences exist between the three conditions of quality of indicators and two conditions of number of indicators. Examination of the effect of the study factors on the variability of standard 
error bias of latent class CRPs shows that only the interaction effect of number of indicators, 
quality of indicators, level-1 covariate effect, and level-2 covariate effects demonstrates medium effects on explaining the variability of the latent class 1 CRPs™ standard error bias.  The effects of the study factors can be seen to respond differently to different patterns in latent classes. Latent class 1 and latent class 3 have similar patterns (i.e., same CRPs across all 
indicators), while latent class 2 has mixed pattern (i.e., half of the indicators have high CRPs and 
the other half of the indicators have low CRPs). For latent class 1 CRP, both the number of the 
quality of indicators has strong effect on the variability of the bias and 95% CI coverage rate. Similarly, quality of indicators has strong effect on explaining the variability of the bias and 95% 
CI coverage rate of latent class 3 CRP. However, for latent class 2, with the mixed pattern in the class, the number of indicators is more important for explaining the variability of the biases and 95% CI coverage rate. The simulation results showed that conditions with 12 indicators generally 
showed smaller biases than those with 6 indicators conditions. This finding is in line with the 
study expectations and is due to the fact that the increasing amount of heterogeneity results from 
the increasing number of indicators. Therefore, the effect of the number of indicators tends to be more important for explaining the variability of latent class 2 CRP.  The implication of this finding is discussed below:  First, when the purpose of applying NP-MLCA is for classification accuracy, lower-level classification accuracy is acceptable when the quality of indicators is 0.8 and the number of 113  indicators is 12. However, when the separation of the higher-level latent classes was not far away (e.g., the separation of two classes are 0.088 in logit scale in this dissertation), this model requires further study to enable it to be applied to the higher-level classification membership for practical use.  Second, when the purpose of applying conditional NP-MLCA is to explore the relationship between the covariates and the latent class variables, the estimates (i.e., slope) can 
be extremely biased and not trustworthy when the quality of indicators is 0.6. This is especially 
the case when only 6 indicators are included, and despite which level the covariates are 
incorporated into. In other words, researchers should avoid the conditions where quality 
indicators are 0.6 and number of indicators is 6 when applying the model, especially if the composition of the latent classes forms the research focus. Third, when covariate effect at the higher-level is the main area of interest to the researcher,  sample size can compensate for the lower quality of indicators, but only for the 
quality of indicators 0.7. Both the effect of number of groups and group sizes can account for the 
total variability of the biases and 95% CI coverage rates. However, if researchers are allowed to choose one between number of groups and group sizes, improving the number of groups, in combination with the quality of indicators to 0.8, will help to decrease the biases at a faster rate.  Fourth, when the estimates of latent classes™ CRPs do not show homogeneous patterns across indicators, the number of indicators matters for the accuracy of the estimates. The quality of estimation is better when 12 indicators are involved. For application studies, this means that 
researchers can be less worried about situations where the identified latent class was not 
homogeneous when applying this method. The results of this dissertation provide evidence that 114  increasing the number of indicators can help with the quality of the estimates. However, the simulation scenario described in this dissertation only investigated three kinds of heterogeneous pattern, even though different heterogeneous patterns may show in practice. Simulation studies regarding different scenarios of heterogeneity can be explored further in a future research project.   Fifth, like multiple indicators and multiple causes (MIMIC) model that incorporates the covariates in the factor analysis to examine the relationship between the factor scores and demographic covariates, 1-step procedure LCA replaced continuous latent variable (i.e., factor) in MIMIC model with the categorical latent variable (i.e., latent class variables). Although the 1-step model was criticized because the incorporation of different covariates results in variance of 
individual™s latent class membership solution, the results shown in this dissertation reveals that 
the covariate effect does not have a strong meaningful impact on the variance of classification 
accuracy at both levels. This implication may help lessen application researchers™ concern when using conditional NP-MLCA for classificatory purposes.  7.2 Summary of the Empirical Study The empirical study consisted of data from 289 sites and a total of 6,580 individuals. Individuals could be categorized into one of three groups: improved, non-improved, and mixed. Based on the distributions of individual latent classes of each site, sites were categorized into one 
of two groups: Type I sites and Type II sites. Type I sites have a higher proportion of improved groups, while Type II sites have a higher proportion of non-improved groups.  The results of the empirical study showed that the quality of indicators are greater than 0.85, or less than 0.15 for improved group and non-improved group respectively, but the quality of indicators in the mixed group were somewhat low {0.6, 0.4} or {0.7, 0.3}. Given the sample 115  size and the level-1 covariate effect in this study, the earlier simulation results suggest that the estimates of the CRPs in the three latent classes at the individual level are trustworthy, as well as the level-1 covariate effects comparing non-improved group with improved group. However, since CRPs in the mixed group are 0.6, the simulation showed that the incorporated level-1 covariate effects are generally biased. The simulation result implied that the estimates of the effects of covariates on the mixed group compared with the improved group may not be accurate. To address this shortcoming, the simulation results suggest an increase in the number of indicators and the group size would improve model performance.  7.3 Discussion  Researchers within the field of latent class analysis have different views regarding how covariates should be incorporated. Many think that the purpose of incorporating the covariates is to investigate the composition of the latent classes and therefore suggest that they should be 
examined after individual latent class membership was obtained. This 3-step procedure was broadly applied in application studies. However, application researchers need to rely on the 
software to correct the estimates of the covariate effects. Otherwise, the covariate effects are systematically biased.  Instead of examining the covariate effects in later steps, the model estimation process for 1-step procedure estimates the model and the covariate effect simultaneously. This means that researchers can argue that the estimated latent class membership, no matter what the highest posterior probability was or if pseudo-classification method was applied to assign latent class membership for each subject, the posterior probability that these assigning methods used are 
conditional probabilities that conditioning on incorporated covariates, unlike the unconditional posterior probabilities utilized in the 3-step procedure. This explains the classification solution 116  inconsistency between conditional and unconditional NP-MLCA applied using a real dataset in Chapter 6, and which was also criticized in previous research (Bolck et al., 2004).  However, the results of the simulation studies included in this dissertation shows that the difference between level-1 covariate effects and level-2 covariate effects do not feature strongly in accounting for variation in classification accuracy. In other words, no large meaningful difference exists between different effects of covariates in terms of classification accuracy. There 
is no evidence to support the notion that the total variance of classification accuracy, whether at 
the individual or group level can be accounted for as resulting from covariate effects. This finding advances the pre-existing knowledge of multi-level mixture regression modeling, with previous simulations having fixed the covariate effect at one (Finch & French, 2013).  Number of indicators and quality of indicators has a positive association with classification accuracy. In Finch & French (2013), the number of indicators is important for classification accuracy. When the number of indicators is 15, it resulted in greater classification accuracy at both levels in NP-MLCA compared to that when the number of indicators is 5 or 10. However, the effect of quality of indicators was ignored in previous multilevel mixture research. 
Researchers tend to fix the quality of indicators as 0.8 and higher for better classification solution. 
In application research, a high quality of indicators was not always available. This dissertation extended the results to more study factors, providing the evidence that the quality of the indicators is also an essential factor featuring in classification accuracy.  Previous studies using factor analysis (Marsh et al., 1998) and standard latent class analysis (Wurpts & Geiser, 2014) showed that high-quality indicators were beneficial for parameter estimation and could compensate for the small effects of the covariates. In the 117  simulation studies within this dissertation, the interaction effects among study factors were thoroughly investigated by including the number of indicators, number of groups and group sizes. 
As a study is in a multilevel context, the quality of parameter estimation is expected to be more 
complicated and to have a function within a series of factors. Large sample size, greater number of indicators, and high quality of indicators are beneficial to the quality of parameter estimation 
of the structural part of the model (i.e., the level-1 / level-2 covariate effects), while all study factors, except large sample size, can improve the quality of parameter estimation of the measurement part of the model (i.e., latent class conditional response probabilities).  In this dissertation, when the condition involves 12 indicators, large sample size, quality of indicators 0.8, and all of level-1 and level-2 covariates are 1, the results of parameter estimation recovery of level-1 covariates and level-2 covariates is similar to the results found within a previous study (Finch & French, 2013) for small bias and high coverage rates. This dissertation extends the results for varied level-1 and level-2 covariates. When the covariate effects increase, the parameter estimation of level-1 covariate on latent class 1 remains good; however, the parameter estimation of the level-2 covariate showed increasing bias along with the increasing magnitude of the level-2 covariate effects. The bias showed within an acceptable range only when the level-2 covariate effect was set to be one. Odds ratio = 1 for level-2 covariate effect means no effect of level-2 covariate on the level-2 latent class variable. In other words, even though the 95% CI coverage rate is in an acceptable range across the different levels of covariate effects, the bias of level-2 covariate effect is acceptable only when the covariate has no effect at the higher level.  Previous study also showed that the inclusion of 15 indicators was related to a higher coverage rate of level-2 covariate (Finch & French, 2013) than 5 or 10 indicators. This study 118  differed from the design of the simulation work conducted by Finch & French™s (2013) as the quality of indicators was included as one of the study factors. The results showed that when the quality of indicators is included, the main effect of quality of indicators (=0.43) and interaction effect of number of indicators and quality of indicators (=0.17 ) showed a stronger association with higher coverage rates than the number of indicators (=0.03).  The difference between the research conducted for this dissertation and that of the previous study also indicated that the results of the simulation work strongly relate to the design of the simulation and the conditions involved within each simulation design. For example, the heterogeneity introduced in the multilevel regression mixture study in Muthén & Asparouhov's 
(2009b) simulation work did not only result from traditional random intercept variances but also from random slope variances. They included level-2 covariates, which were incorporated in the measurement part of the model instead of the structural part of the model. Since the definition of mixture can vary in terms of the latent class itself or the relationship between covariates and 
latent classes, when it was applied in multilevel context, the group dependencies interacted with 
the heterogeneity, which makes the results of simulation studies in mixture models difficult to 
compare. Therefore, when the design of the simulation work is not the same or nearly 
comparable, the interpretation of the results from simulation work should be considered more carefully.   7.4 Limitation There are several limitations of this dissertation. First, in the design of simulation study, the real data set scenario is approximated. In order to make the generated data close to the real data, the distribution of two random log-odds 119  (i.e., schools) is assumed to be normal, but in the model estimation, the conventional non-parametric approach proposed in Asparouhov & Muthén (2008) and Vermunt (2003) were applied for the model estimation. If it were to use the nonparametric approach to generate the data, the data generated would have had only two kinds of schools and six kinds of posterior 
probability patterns, which is not close to the real scenario. Therefore, to approximate the real data scenario, the discrepancy between the population model and the analysis model is a shortcoming that cannot be overcome within this dissertation.  Model misspecification is going to be an issue every time a nonparametric approach is applied in MLCA. This is due to the fact that a discrete random component is utilized to approximate the random component. It is similar to the situation where a continuum scale is categorized to a number of levels for a variable. The power will drop, but how much it drops 
depends on how much information is lost during the categorizing process and how great the 
discrepancy is from the original distribution. Even though the purpose of the nonparametric approach proposed by Vermunt (2003) is, in the first instance, to decrease the computation intensity, to what extent the information removed during the process of utilizing this approach has on affecting the final estimation is a sensitivity question. It can be expected that, if the model 
estimation results are not sensitive to the misspecification, this approach is going to gradually replace a parametric approach for its flexibility of model specificity. However, the sensitivity is not covered as an area of examination within this dissertation. This dissertation can only represent one kind of misspecification, with the results and findings of 
this dissertation being also limited to this type of misspecification. There are different ways to 
generate the population dataset for the sensitivity of the misspecification. One possible way is to 
vary different kinds of distribution for random components for the population data and apply the 120  NP-MLCA to evaluate the estimation. The other way is to vary the separation of the mixture components at the different levels in terms of their means and variance covariance matrix.  Second, with regards to the result, even though classification accuracy is not greatly affected by the covariate effects, it is worth noting that the group-level classification accuracy is only around 0.5 across all conditions. This is due to the small separation of two group-level latent classes specified in the population model, where the two specified latent classes in the population were not separate enough (two means of random log-odds are fixed at: 0.061 and          -0.027 relatively). In other words, in a situation where the heterogeneity is not obvious, it is reasonable to say that the NP-MLCA model has difficulty in identifying groups™ correct latent class membership. As the NP-MLCA model is currently utilized for classification purposes at both individual and group level in application studies, this suggests that future methodological 
work on examining the relationship between the degree of heterogeneous population and the 
classification accuracy is necessary in order to improve upon the findings presented in this dissertation.  Third, in the simulation study, it is assumed that the covariates are all unidimensional, with no unmeasured covariates affecting the latent class membership, and no interaction between 
the level-1 covariate and the level-2 covariate. However, from another perspective, these assumptions can be viewed as a limitation of this study. Since the real world data are rarely as simple as in the design of the data structure presented here, future studies within this field will be essential in order to free these assumptions.  Fourth, for the sake of simplicity, the model selected in the dissertation is a random intercept model in MLCA, with the covariate effect only being applied to the structural part of 121  the model instead of the measurement part. In other words, the covariate effects only have an impact on the variance of the latent class probability rather than the conditional response 
probabilities of the model. This may not be the perfect way to represent the heterogeneity of the 
real population. Future studies are necessary to explore different kinds of variety in terms of heterogeneity.  Fifth, the latent class indicators are dichotomized variables in the model. It can be argued that the importance of quality of indicators in the results may be eased when the indicators have more than two categories involved or when the indicators follow other types of distribution.  Sixth, this dissertation research was conducted using Mplus. Mplus allows users to specify population parameters for date generation purposes and analysis models for estimation purposes in the same input file. By specifying the population parameters in the model population 
section in Mplus syntax, the process can be criticized for not being transparent enough in terms of simulation procedure. Unless the researchers undertake post-hoc checks on the generated dataset, it can be difficult to tell if the generated dataset corresponds to what the researchers 
designed. Although, for the purposes of this dissertation, the simulated data had been checked 
before the analysis was run, researchers are increasingly suggesting using different software for the data generation procedure and for the model estimation procedure. Since the generated 
dataset cannot be fully trusted if the procedure is not transparent, this may be another limitation of this dissertation. Seventh, large proportion of replications has singularity and non-positive definite problems in 1-step procedure model estimation. Even though 1-step covariate-incorporating procedure avoids systematic biased estimate issues, one of the disadvantages of this procedure is 122  that the model estimation process tends to encounter singularity of the information matrix and non-positive definite problems. Two scenarios would trigger these warning messages: the model is not identified or the cells are sparse. When the covariates are incorporated, the model 
estimation and the estimation of the covariate effect are in the iterative process together. 
Singularity problems mostly resulted from empty cells in the joint distribution of latent class 
variables and independent variables and small sample sizes (number of groups in particular), 
with fewer indicators exacerbating the problem. In Mplus the standard errors were fixed to near zero when these problems occurred.  7.5 Future Studies  This study advanced knowledge in the field of MLCA by investigating how classification-error and parameter estimates changes among 384 combinations of the number and quality of indicators, the covariate effects, and sample sizes on two levels. The benefits of the proposed research are both methodological and practical. First, the results of this study can be expanded to multilevel latent transition analysis (MLTA) or multilevel mover-stayer model with incorporated fixed or time-varying covariates when multiple time points or longitudinal data are available. The probability of a higher-unit transit to the same or a different latent class in the adjacent time point can be investigated. The 
transition probabilities can also be modeled, especially at the individual level. The transition 
matrix at the lower level can be compared with the higher-level transition matrix to ascertain shifting patterns among latent classes.  Second, the findings of this study can be applied in intervention studies. The identified latent classes on the higher level and the lower level can be used as the strata in the propensity-123  score stratification method. Compared to propensity-score stratification, the advantage of using the identified latent classes as the strata at the different levels is that it takes into consideration the probability of item responses (i.e., the measurement model) in the model, which makes a more convincing statement for the stratification method. Third, the findings of this study can be adapted to contexts in which the treatment is heterogeneous and random assignment is not available. To solidly identify homogenous treatment groups, it is important to choose good-quality measured indicators and covariates, to the extent that the availability of the data allows. Previous studies applied LCA to identify the latent class with homogenous treatment, treating the estimated latent-class membership as the observed treatment-group membership and then applying propensity-score methods to control the effects of confounding variables when estimating the treatment effect (Schuler, Leoutsakos, & Stuart, 2014). However, this procedure ignored classification errors in the propensity-score-weighting step and is not suitable if the treatment is administered at the higher level. In many fields such as organizational psychology, education, and health science, it is common for data to 
occur in nested structures. Therefore, how covariates can be incorporated into NP-MLCA and how they may interact with the measured indicators is worthy of comprehensive examination. Fourth, taking advantage of the model-based approach, MLCA with covariates allows researchers to use goodness-of-fit indices for model selection. More flexibility under the latent variable modeling framework can be explored with the findings of this study. For example, by conducting multiple-group analysis, we can answer some interesting questions such as whether the relationship between taking STEM classes and being in a high-achieving school is the same for males and females, or whether the effects of staff satisfaction on school performance differs 
between public schools and private schools. In addition, by relaxing the assumption that item 124  response parameters are equal across the higher-level unit, we can examine how subjects respond to the measured indicators differently, which can be referred to different item functioning (DIF). Furthermore, by replacing the measurement model of MLCA, researchers can integrate different measurement models within the model, for example, an IRT model. Fifth, different latent-class assignment methods may result in differing degrees of classification accuracy. In LCA, two methods have been commonly used: modal assignment (Nagin, 2005, p. 115) and pseudo-class assignment (Bandeen-Roche et al., 1997). In the former method, latent-class membership is assigned based on the highest posterior probability for class membership. In the latter, latent-class membership is predicted via two steps: 1) using the estimated posterior class membership probability to define the random draw from the 
multinomial distribution; and 2) performing the first step multiple times, and combining the 
results across draws using Rubin™s rule for multiple imputation (Rubin, 1987). Previous study (Schuler et al., 2014) showed that when entropy is low and the covariate effects are large, parameter estimations resulting from pseudo-class assignment tend to have a higher percentage of bias than those from modal assignment. However, it is still unknown if covariates at the different levels in NP-MLCA matter in the same way.   125      
    APPENDICES  126  Appendix A Template Mplus Syntax title:  monte carlo for two-level LCA with categorical  latent class indicators  montecarlo:  names are u1-u${ni} x w;     cutpoints = x(0) w(0);  nobservations = ${nobs}; ! nobs == nc*cs  ncsizes = 1;  csizes = ${nc} (${cs});  ! num cluster, cluster size  generate = u1-u${ni}(1);  categorical = u1-u${ni};  genclasses = cb(2 b) cw(3);  classes = cb(2) cw(3);  within = x u1-u${ni};  between = w cb;  seed = 3454367;  nrep = ${nr};     repsave = all;  save = ${id}_*.dat;  analysis:  type = twolevel mixture;     starts = 20 5;  adaptive = off;  integration = 50;  model population:   %within%  %overall%   x@1;   [x@0];  
 [cw#1*0.061 cw#2*-0.027];  
 cw#1 on x*${l1p1};  cw#2 on x*${l1p2};   %between%  %overall%    [w@0]; w@1;      cw#1 on cb#1*.75 ;      cw#2 on cb#1*.5 ;     cb on w*${l2p1};      [cb#1*0];  127        cw#1*0.727;       cw#2*0.435;  model population-cw:      %within%  %cw#1%  [u1$1-u${ni}$1*${qi}];   %cw#2%  [u1$1-u${ni2}$1*${qi}];  ! ni2 == ni/2      [u${ni3}$1-u${ni}$1*-${qi}];  ! ni2 == ni/2+1   %cw#3%  [u1$1-u${ni}$1*-${qi}];   model:   %within%  %overall%  [cw#1*0.061 cw#2*-0.027];  cw#1 on x*${l1p1};  cw#2 on x*${l1p2};   %between%  %overall%  cw#1 on cb#1*.75 ;              cw#2 on cb#1*.5 ;     cb on w*${l2p1};      [cb#1*0];   model cw:      %within%  %cw#1%  [u1$1-u${ni}$1*${qi}];  
 %cw#2%  [u1$1-u${ni2}$1*${qi}];      [u${ni3}$1-u${ni}$1*-${qi}];   %cw#3%  [u1$1-u${ni}$1*-${qi}];  output:  tech9;   128  Appendix B Descriptive Tables Table 19. Individual-Level Classification Accuracy by Study Factors Number  of Indicators Level-1  Covariate Effect Level-2  Covariate Effect Number of Groups = 50 Number of Groups = 150 Group Size = 30 Group Size = 60 Group Size = 30 Group Size = 60 Quality of Indicators Quality of Indicators Quality of Indicators Quality of Indicators    0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 6 (1, 1) 1 0.35 0.42 0.58 0.36 0.44 0.55 0.35 0.44 0.60 0.36 0.40 0.60 1.33 0.35 0.41 0.57 0.36 0.44 0.53 0.35 0.42 0.60 0.36 0.40 0.62 1.5 0.36 0.42 0.57 0.35 0.45 0.54 0.35 0.42 0.59 0.36 0.42 0.60 1.75 0.35 0.43 0.56 0.35 0.45 0.59 0.35 0.43 0.59 0.35 0.43 0.59 (1.33, 2) 1 0.34 0.43 0.56 0.36 0.45 0.57 0.35 0.44 0.61 0.39 0.38 0.60 1.33 0.35 0.42 0.53 0.37 0.46 0.54 0.36 0.43 0.61 0.37 0.43 0.60 1.5 0.34 0.43 0.54 0.37 0.44 0.54 0.35 0.43 0.59 0.38 0.39 0.60 1.75 0.35 0.43 0.54 0.37 0.44 0.57 0.35 0.44 0.60 0.38 0.38 0.59 (1.5, 3) 1 0.36 0.42 0.54 0.36 0.42 0.56 0.36 0.43 0.56 0.39 0.41 0.60 1.33 0.36 0.42 0.55 0.37 0.43 0.58 0.35 0.42 0.54 0.39 0.41 0.61 1.5 0.35 0.42 0.50 0.38 0.40 0.58 0.36 0.45 0.57 0.40 0.41 0.60 1.75 0.35 0.42 0.50 0.37 0.42 0.56 0.35 0.44 0.56 0.39 0.42 0.58 (1.75, 5) 1 0.36 0.45 0.54 0.38 0.44 0.55 0.38 0.44 0.56 0.40 0.40 0.61 1.33 0.37 0.45 0.54 0.38 0.44 0.56 0.40 0.44 0.54 0.40 0.41 0.59 1.5 0.36 0.46 0.53 0.38 0.44 0.55 0.40 0.46 0.53 0.41 0.42 0.59 1.75 0.37 0.46 0.54 0.38 0.44 0.52 0.40 0.45 0.58 0.42 0.41 0.57    129   Table 19. (cont™d) Number  of Indicators Level-1  Covariate Effect Level-2  Covariate Effect Number of Groups = 50 Number of Groups = 150 Group Size = 30 Group Size = 60 Group Size = 30 Group Size = 60 Quality of Indicators Quality of Indicators Quality of Indicators Quality of Indicators    0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 12 (1, 1)  1 0.33 0.46 0.79 0.36 0.51 0.78 0.35 0.52 0.80 0.35 0.52 0.80 1.33 0.34 0.44 0.79 0.35 0.51 0.78 0.35 0.54 0.80 0.34 0.55 0.80 1.5 0.34 0.44 0.79 0.35 0.52 0.77 0.34 0.53 0.80 0.35 0.56 0.80 1.75 0.35 0.45 0.78 0.35 0.51 0.79 0.35 0.55 0.80 0.35 0.54 0.80 (1.33, 2) 1 0.34 0.51 0.77 0.36 0.54 0.80 0.37 0.58 0.79 0.37 0.58 0.80 1.33 0.34 0.49 0.77 0.35 0.55 0.80 0.36 0.56 0.80 0.37 0.58 0.81 1.5 0.34 0.48 0.76 0.36 0.55 0.80 0.36 0.55 0.80 0.36 0.59 0.81 1.75 0.34 0.51 0.77 0.37 0.54 0.80 0.35 0.55 0.80 0.35 0.59 0.81 (1.5, 3) 1 0.34 0.51 0.75 0.37 0.58 0.78 0.37 0.59 0.79 0.38 0.61 0.81 1.33 0.35 0.51 0.77 0.36 0.59 0.80 0.37 0.60 0.79 0.38 0.62 0.81 1.5 0.35 0.50 0.75 0.36 0.58 0.80 0.36 0.60 0.80 0.39 0.61 0.81 1.75 0.34 0.51 0.75 0.37 0.57 0.80 0.36 0.60 0.79 0.39 0.62 0.81 (1.75, 5) 1 0.38 0.54 0.72 0.39 0.62 0.76 0.39 0.62 0.80 0.41 0.64 0.74 1.33 0.36 0.56 0.74 0.39 0.63 0.78 0.39 0.62 0.79 0.40 0.64 0.77 1.5 0.37 0.58 0.75 0.40 0.61 0.76 0.38 0.61 0.78 0.41 0.65 0.78 1.75 0.36 0.57 0.74 0.40 0.62 0.76 0.39 0.61 0.77 0.41 0.64 0.77   130   Table 20. Group-Level Classification Accuracy by Study Factors Number  of Indicators Level-1  Covariate Effect Level-2  Covariate Effect Number of Groups = 50 Number of Groups = 150 Group Size = 30 Group Size = 60 Group Size = 30 Group Size = 60 Quality of Indicators Quality of Indicators Quality of Indicators Quality of Indicators    0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 6 (1, 1) 1 0.50 0.53 0.55 0.50 0.54 0.56 0.51 0.53 0.56 0.50 0.55 0.57 1.33 0.51 0.54 0.55 0.51 0.55 0.56 0.51 0.52 0.56 0.50 0.54 0.58 1.5 0.51 0.54 0.55 0.50 0.55 0.57 0.52 0.53 0.56 0.51 0.54 0.57 1.75 0.52 0.54 0.55 0.50 0.55 0.57 0.52 0.53 0.55 0.50 0.56 0.58 (1.33, 2) 1 0.50 0.53 0.54 0.50 0.55 0.55 0.50 0.53 0.54 0.51 0.55 0.55 1.33 0.51 0.52 0.54 0.51 0.54 0.56 0.51 0.53 0.54 0.50 0.55 0.55 1.5 0.51 0.52 0.55 0.50 0.54 0.56 0.51 0.53 0.54 0.52 0.54 0.55 1.75 0.51 0.53 0.55 0.50 0.53 0.56 0.51 0.53 0.54 0.51 0.54 0.54 (1.5, 3) 1 0.50 0.52 0.55 0.51 0.53 0.55 0.51 0.54 0.54 0.51 0.53 0.54 1.33 0.50 0.50 0.54 0.51 0.53 0.55 0.50 0.53 0.54 0.51 0.55 0.53 1.5 0.51 0.50 0.54 0.50 0.52 0.55 0.51 0.53 0.54 0.51 0.54 0.53 1.75 0.51 0.51 0.55 0.50 0.53 0.55 0.51 0.52 0.54 0.51 0.53 0.54 (1.75, 5) 1 0.51 0.52 0.55 0.51 0.54 0.55 0.50 0.51 0.53 0.51 0.52 0.54 1.33 0.51 0.50 0.54 0.52 0.54 0.54 0.51 0.51 0.53 0.51 0.52 0.54 1.5 0.51 0.51 0.53 0.51 0.53 0.55 0.52 0.52 0.53 0.50 0.53 0.54 1.75 0.52 0.51 0.53 0.52 0.53 0.54 0.52 0.51 0.53 0.51 0.53 0.53    131  Table 20.  (cont™d) Number  of Indicators Level-1  Covariate Effect Level-2  Covariate Effect Number of Groups = 50 Number of Groups = 150 Group Size = 30 Group Size = 60 Group Size = 30 Group Size = 60 Quality of Indicators Quality of Indicators Quality of Indicators Quality of Indicators    0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 12 (1, 1)  1 0.51 0.48 0.50 0.51 0.48 0.49 0.50 0.46 0.51 0.50 0.47 0.49 1.33 0.50 0.48 0.51 0.51 0.47 0.47 0.50 0.47 0.51 0.49 0.49 0.50 1.5 0.49 0.49 0.51 0.51 0.48 0.48 0.49 0.47 0.51 0.50 0.50 0.51 1.75 0.50 0.45 0.50 0.50 0.48 0.49 0.50 0.46 0.51 0.49 0.47 0.50 (1.33, 2) 1 0.50 0.49 0.49 0.50 0.50 0.50 0.50 0.48 0.52 0.50 0.47 0.49 1.33 0.49 0.48 0.50 0.49 0.50 0.47 0.50 0.48 0.50 0.50 0.49 0.48 1.5 0.50 0.48 0.49 0.50 0.49 0.49 0.50 0.49 0.50 0.50 0.48 0.49 1.75 0.49 0.50 0.49 0.50 0.50 0.50 0.50 0.49 0.49 0.50 0.49 0.49 (1.5, 3) 1 0.50 0.49 0.49 0.51 0.50 0.46 0.50 0.47 0.51 0.49 0.47 0.51 1.33 0.51 0.49 0.48 0.49 0.50 0.47 0.49 0.48 0.51 0.51 0.47 0.50 1.5 0.50 0.49 0.49 0.50 0.49 0.48 0.49 0.49 0.50 0.51 0.48 0.51 1.75 0.50 0.48 0.49 0.50 0.49 0.49 0.49 0.49 0.51 0.51 0.49 0.51 (1.75, 5) 1 0.49 0.50 0.51 0.51 0.49 0.47 0.50 0.47 0.50 0.51 0.47 0.49 1.33 0.49 0.49 0.48 0.50 0.49 0.48 0.50 0.47 0.51 0.50 0.49 0.49 1.5 0.49 0.49 0.50 0.50 0.48 0.48 0.50 0.47 0.51 0.50 0.49 0.50 1.75 0.49 0.52 0.49 0.51 0.48 0.48 0.50 0.48 0.51 0.49 0.49 0.49    132  Table 21. Bias of Level-1 Covariate Effect on Latent Class 1 by Study Factors Number  of Indicators Level-1  Covariate Effect Level-2  Covariate Effect Number of Groups = 50 Number of Groups = 150 Group Size = 30 Group Size = 60 Group Size = 30 Group Size = 60 Quality of Indicators Quality of Indicators Quality of Indicators Quality of Indicators    0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 6 (1, 1) 1 -0.19 -2.56 -0.03 0.04 -0.10 -0.02 0.09 -0.02 0.00 -0.16 0.00 0.00 1.33 0.02 -0.65 -0.03 -1182.08 -0.16 -0.03 -1.26 0.03 -0.01 -0.38 0.00 0.01 1.5 0.18 -0.27 -0.02 -1.09 -0.15 -0.03 -1.25 0.03 -0.01 -0.08 0.03 0.01 1.75 0.41 -0.28 0.01 -1.01 -0.12 -0.02 -0.83 -0.01 0.00 -2.70 0.01 0.01 (1.33, 2) 1 1.04 -4.16 -0.12 17.40 -0.56 -0.18 -0.54 -0.13 -0.14 -1.03 -0.32 -0.11 1.33 1.03 -0.50 -0.13 -1.51 -0.20 -0.16 -86.64 -0.15 -0.19 -2.25 -0.26 -0.15 1.5 1.68 -0.39 -0.15 -251.38 -0.30 -0.20 -54.54 -0.17 -0.20 -0.18 -0.31 -0.22 1.75 7.21 -0.11 -0.16 -202.64 -0.32 -0.20 -161.11 -0.23 -0.18 0.74 -0.39 -0.17 (1.5, 3) 1 -82.59 0.09 -0.31 -37.05 -0.63 -0.30 53.49 -0.58 -0.27 0.03 -0.55 -0.20 1.33 -596.09 -0.43 -0.19 1187.52 -0.46 -0.37 1.83 -0.20 -0.36 0.05 -0.58 -0.22 1.5 -149.05 -0.31 -0.31 -733.52 -0.43 -0.38 2.25 -0.26 -0.26 0.01 -0.46 -0.17 1.75 175.21 -0.27 -0.28 -1.40 -0.43 -0.36 -0.25 -0.40 -0.25 -0.42 -0.48 -0.18 (1.75, 5) 1 -0.51 -0.48 -0.27 293.34 -0.28 -0.36 -325.42 -0.37 -0.36 -0.11 -0.86 -0.18 1.33 31.27 -1.11 -0.31 486.91 -0.51 -0.44 26.17 -0.69 -0.40 16.66 -0.78 -0.30 1.5 1.00 -0.85 -0.34 150.35 -0.63 -0.55 -151.79 -0.44 -0.50 -0.55 -0.60 -0.26 1.75 10.27 1.92 -0.32 9.44 -0.48 -0.55 -32.48 1.84 -0.38 -1.65 -0.95 -0.30     
  133  Table 21.  (cont™d) Number  of Indicators Level-1  Covariate Effect Level-2  Covariate Effect Number of Groups = 50 Number of Groups = 150 Group Size = 30 Group Size = 60 Group Size = 30 Group Size = 60 Quality of Indicators Quality of Indicators Quality of Indicators Quality of Indicators    0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 12 (1, 1) 1 5.89 0.06 0.02 -0.94 0.00 -0.01 0.37 0.01 0.03 0.05 -0.02 0.00 1.33 4.33 0.09 0.02 -0.64 -0.01 -0.01 0.37 0.01 0.03 0.15 -0.01 0.00 1.5 1.27 0.03 0.02 -1.11 0.00 0.00 -0.11 -0.01 0.03 0.09 -0.01 0.00 1.75 0.55 0.07 0.02 -1.48 0.00 -0.01 0.26 0.02 0.03 0.23 0.00 0.00 (1.33, 2) 1 0.42 0.03 0.00 0.46 -0.18 -0.03 0.46 -0.08 0.01 0.01 -0.14 -0.03 1.33 2.16 -0.05 -0.02 0.50 -0.17 -0.03 0.36 -0.12 0.01 -0.11 -0.14 -0.03 1.5 12.47 -0.06 -0.02 0.42 -0.19 -0.02 0.06 -0.08 0.01 -0.07 -0.15 -0.03 1.75 517.84 -0.05 -0.02 1.31 -0.18 -0.03 0.16 -0.05 0.01 0.85 -0.15 -0.03 (1.5, 3) 1 20.38 -0.19 -0.02 182.94 -0.15 -0.04 0.82 -0.09 0.00 -0.09 -0.19 -0.03 1.33 0.62 -0.19 0.00 0.80 -0.22 -0.03 0.55 -0.05 0.00 0.12 -0.14 -0.04 1.5 -11.92 -0.18 -0.01 40.62 -0.19 -0.03 0.00 -0.11 0.00 0.01 -0.15 -0.04 1.75 3.23 -0.19 -0.01 0.95 -0.20 -0.02 0.01 -0.10 0.00 0.01 -0.12 -0.03 (1.75, 5) 1 1.16 -0.11 -0.01 5.05 -0.19 -0.01 0.74 -0.12 0.01 0.29 -0.16 -0.02 1.33 -0.78 0.29 0.00 2.40 -0.20 -0.03 0.07 -0.18 -0.01 0.25 -0.20 -0.03 1.5 -0.34 0.14 -0.02 2.11 -0.19 0.00 0.29 -0.25 -0.01 0.21 -0.23 -0.02 1.75 4.29 0.10 0.02 7.75 -0.19 0.01 0.73 -0.20 0.02 0.22 -0.20 -0.02    134  Table 22. Bias of Level-1 Covariate Effect on Latent Class 2 by Study Factors Number  of Indicators Level-1  Covariate Effect Level-2  Covariate Effect Number of Groups = 50 Number of Groups = 150 Group Size = 30 Group Size = 60 Group Size = 30 Group Size = 60 Quality of Indicators Quality of Indicators Quality of Indicators Quality of Indicators    0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 6 (1, 1) 1 -13.96 -12.55 0.00 0.13 -0.90 -0.03 -0.24 -0.03 0.00 -16.15 0.02 -0.01 1.33 46.40 -3.16 -0.01 -131.84 -0.56 -0.03 37.08 -0.01 0.00 -1.78 0.00 0.00 1.5 20.08 -3.13 -0.01 -3.06 -0.53 -0.02 -9.01 -0.42 0.00 -2.55 0.03 0.00 1.75 292.40 -4.57 0.02 -17.01 -0.42 -0.02 17.90 -0.23 0.00 -4.18 0.03 0.00 (1.33, 2) 1 34.11 0.07 -0.19 19.84 -0.35 -0.28 2.05 -0.12 -0.19 0.39 -0.38 -0.16 1.33 65.80 0.05 -0.21 -1.93 -0.25 -0.27 12.96 -0.17 -0.21 17.01 -0.32 -0.20 1.5 94.20 0.16 -0.22 -0.65 -0.33 -0.28 20.23 -0.23 -0.22 90.66 -0.36 -0.27 1.75 183.75 -3.89 -0.24 49.64 -4.10 -0.26 289.38 -0.20 -0.21 414.74 -0.47 -0.23 (1.5, 3) 1 -79.51 0.09 -0.47 17.14 -0.64 -0.41 74.35 -0.67 -0.43 -35.79 -0.72 -0.38 1.33 -176.11 -0.33 -0.37 1691.00 -0.56 -0.55 35.40 -0.34 -0.54 0.73 -0.67 -0.43 1.5 -140.91 -0.31 -0.54 216.22 -0.52 -0.56 32.90 -0.40 -0.43 0.50 -0.64 -0.41 1.75 180.16 -1.15 -0.55 -1.31 -0.47 -0.51 29.53 -0.47 -0.44 0.25 -0.67 -0.38 (1.75, 5) 1 2.54 -0.13 -0.64 301.39 -0.58 -0.73 582.44 -0.62 -0.61 1.26 -1.12 -0.57 1.33 34.59 -0.06 -0.66 485.84 -0.61 -0.79 35.35 -0.51 -0.72 17.84 -0.89 -0.63 1.5 4.06 0.08 -0.78 149.46 -0.75 -0.88 16.71 -0.37 -0.72 0.32 -0.86 -0.68 1.75 14.81 3.73 -0.73 13.21 -0.59 -0.89 20.19 1.92 -0.60 0.71 -1.10 -0.76    135  Table 22. (cont™d) Number  of Indicators Level-1  Covariate Effect Level-2  Covariate Effect Number of Groups = 50 Number of Groups = 150 Group Size = 30 Group Size = 60 Group Size = 30 Group Size = 60 Quality of Indicators Quality of Indicators Quality of Indicators Quality of Indicators    0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 12 (1, 1) 1 -1.77 0.08 0.00 -0.38 0.00 0.00 0.29 0.00 0.01 0.04 0.00 0.00 1.33 1.23 0.09 0.01 -1.15 -0.01 0.00 0.18 0.00 0.01 0.21 0.00 -0.01 1.5 1.02 0.01 0.00 -1.16 0.00 0.00 -0.08 -0.01 0.01 0.21 0.00 -0.01 1.75 41.32 0.07 0.00 -1.89 0.02 -0.01 0.30 0.00 0.01 0.24 0.00 -0.01 (1.33, 2) 1 0.53 -0.06 -0.05 0.64 -0.13 -0.06 0.11 -0.12 -0.04 -0.32 -0.12 -0.05 1.33 2.61 -0.17 -0.05 0.49 -0.15 -0.05 0.09 -0.10 -0.03 -0.32 -0.11 -0.05 1.5 12.10 -0.20 -0.06 0.32 -0.12 -0.05 0.28 -0.10 -0.03 -0.09 -0.12 -0.05 1.75 517.92 -0.16 -0.06 0.22 -0.14 -0.05 -0.11 -0.10 -0.03 0.58 -0.13 -0.05 (1.5, 3) 1 17.41 -0.15 -0.10 182.46 -0.12 -0.07 0.89 -0.12 -0.06 -0.39 -0.15 -0.07 1.33 34.04 -0.17 -0.11 0.34 -0.21 -0.08 0.33 -0.10 -0.05 -0.46 -0.14 -0.07 1.5 5.55 -0.26 -0.13 40.10 -0.18 -0.06 -0.19 -0.12 -0.05 -0.32 -0.13 -0.07 1.75 -148.96 -0.28 -0.12 0.45 -0.17 -0.07 -0.01 -0.13 -0.07 -0.43 -0.12 -0.07 (1.75, 5) 1 -0.15 -0.05 -0.32 1.80 -0.13 -0.18 0.71 -0.10 -0.17 -0.32 -0.16 -0.19 1.33 -0.76 0.16 -0.22 2.18 -0.15 -0.12 0.18 -0.17 -0.08 -0.21 -0.21 -0.20 1.5 -1.00 0.09 -0.26 11.98 -0.17 -0.15 9.50 -0.18 -0.12 -0.07 -0.22 -0.22 1.75 -1.52 0.05 -0.20 5.54 -0.18 -0.23 0.51 -0.17 -0.23 -0.34 -0.18 -0.21    136  Table 23. Bias of Level-2 Covariate Effect by Study Factors Number of Indicators Level-1 Covariate Effects Level-2 Covariate Effects Number of Groups = 50 Number of Groups = 150 Group Size = 30 Group Size = 60 Group Size = 30 Group Size = 60 Quality of Indicators Quality of Indicators Quality of Indicators Quality of Indicators    0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 6 (1, 1) 1 460.74 -0.33 -0.49 5.46 0.26 0.23 -371.39 -0.09 -0.02 -0.76 -0.01 -0.03 1.33 66.49 1.81 -99.01 341.54 -0.35 0.63 21.71 6.98 -0.27 -0.73 -0.26 -0.26 1.5 221.82 25.08 -0.29 24.57 -0.11 0.10 1258.45 4.02 -0.37 -6.17 -0.36 -0.38 1.75 242.70 6.56 -0.48 -84.74 -0.37 -0.22 130.07 -0.53 -0.50 -0.72 -0.47 -0.48 (1.33, 2) 1 69.43 1.11 -0.27 -170.89 -0.12 0.27 -488.88 -0.24 -0.06 -0.83 0.06 -0.03 1.33 -0.34 1.41 -0.56 -928.23 0.24 0.02 -54.92 -0.06 -0.34 1954.98 -0.25 -0.30 1.5 -2.72 -17.70 -1.62 -79.47 0.05 -0.06 -647.43 -0.10 -0.41 0.05 -0.33 -0.39 1.75 -167.40 0.46 677.28 60.58 0.15 -0.23 23.98 -0.16 -0.51 -0.67 -0.48 -0.53 (1.5, 3) 1 -16.34 -46.67 -0.55 -5.08 -0.31 -0.10 197.04 27.68 0.04 -7671.29 0.06 -0.05 1.33 -47.00 -128.18 -229.95 2.44 -0.40 538.46 134.52 2.48 -0.25 2.51 -0.24 -0.34 1.5 -13.46 -49.04 -1.00 17.61 -0.46 509.61 223.47 2.43 -0.38 0.80 -0.26 -0.43 1.75 123.14 0.91 -15.87 -10.04 -0.28 -0.31 94.89 0.04 -0.53 0.84 -0.50 -0.54 (1.75, 5) 1 -9.57 -598.99 0.45 -7.24 -0.34 -0.04 -7.28 -33.80 -0.02 -0.99 0.01 -0.05 1.33 -41.65 5.06 0.03 -26.47 -0.81 -0.02 245.83 -0.11 -0.28 -723.76 -0.21 -0.22 1.5 -30.40 9.54 -0.78 16.64 -0.95 -0.13 14.65 -151.97 -0.34 -1.65 -0.31 -0.38 1.75 -4.60 232.77 -0.72 -93.82 -0.97 -0.28 15.91 -58.70 -0.46 -0.84 -0.42 -0.55    137  Table 23. (cont™d) Number  of Indicators Level-1  Covariate Effect Level-2  Covariate Effect Number of Groups = 50 Number of Groups = 150 Group Size = 30 Group Size = 60 Group Size = 30 Group Size = 60 Quality of Indicators Quality of Indicators Quality of Indicators Quality of Indicators    0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 12 (1, 1) 1 106.35 -24.83 0.10 1.33 -0.04 -0.07 -5408.91 -0.04 -0.02 0.44 0.00 -0.01 1.33 -2.81 -15.21 -0.10 -25.37 -0.52 -0.36 -21.50 -0.24 -0.34 -0.06 -0.23 -0.30 1.5 -30.23 -0.82 -0.28 -307.54 -0.69 -0.46 -3.57 -0.43 -0.45 0.09 -0.37 -0.40 1.75 835.40 -0.98 -0.56 -46.34 -0.70 -0.56 -174.31 -0.58 -0.61 -0.48 -0.56 -0.59 (1.33, 2) 1 178.74 0.08 -0.26 -40.85 -0.06 0.38 -486.66 0.01 -0.01 0.14 0.04 0.00 1.33 241.42 17.65 -0.36 -216.39 -0.33 0.01 302.99 -0.24 -0.30 -0.60 -0.26 -0.29 1.5 208.22 8.20 -0.52 90.52 -0.53 -0.05 -3.37 -0.39 -0.41 -0.39 -0.38 -0.42 1.75 66.57 4.32 -0.62 64.54 -0.64 -0.26 -45.25 -0.59 -0.57 -0.74 -0.53 -0.59 (1.5, 3) 1 14.74 -0.08 0.04 -7.63 -0.10 -0.24 -23.47 0.03 -0.01 -0.30 0.09 0.03 1.33 19.19 -0.25 -0.22 -1.84 -0.25 0.07 -0.31 -0.26 -0.32 -0.60 -0.26 -0.28 1.5 6.33 -0.85 -0.36 -377.82 -0.43 -0.62 -1.20 -0.37 -0.41 -0.57 -0.41 -0.41 1.75 -47.79 -1.44 -0.59 -284.87 -0.62 -0.75 -7.43 -0.51 -0.54 -0.71 -0.50 -0.57 (1.75, 5) 1 -3.09 0.10 0.04 -1.26 0.12 -0.03 -570.81 0.06 -0.03 28.74 0.01 0.02 1.33 -60.76 -0.91 -0.29 -1.71 -0.24 -0.46 863.44 -0.27 -0.28 -0.13 -0.27 -0.27 1.5 -62.17 -0.63 -0.46 6.19 -0.54 -0.41 2.19 -0.36 -0.38 -0.26 -0.44 -0.43 1.75 -46.76 -0.46 -0.63 15.92 -0.54 -0.76 -17.21 -0.56 -0.52 -0.38 -0.62 -0.53    138   Table 24. Bias of Conditional Response Probability of Latent Class 1 by Study Factors Number  of Indicators Level-1  Covariate Effect Level-2  Covariate Effect Number of Groups = 50 Number of Groups = 150 Group Size = 30 Group Size = 60 Group Size = 30 Group Size = 60 Quality of Indicators Quality of Indicators Quality of Indicators Quality of Indicators    0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 6 (1, 1) 1 -0.36 -0.14 -0.26 -0.07 -0.30 -0.37 -0.04 -0.20 -0.29 -0.10 -0.47 -0.30 1.33 -0.14 -0.18 -0.23 -0.08 -0.24 -0.42 0.03 -0.30 -0.27 -0.06 -0.45 -0.18 1.5 -0.23 -0.05 -0.36 -0.16 -0.20 -0.41 -0.02 -0.25 -0.32 -0.16 -0.39 -0.24 1.75 -0.27 -0.07 -0.30 0.00 -0.13 -0.32 0.08 -0.21 -0.30 -0.03 -0.31 -0.29 (1.33, 2) 1 -0.12 -0.07 -0.34 -0.24 -0.29 -0.39 0.08 -0.21 -0.25 0.09 -0.44 -0.21 1.33 -0.15 -0.06 -0.45 -0.08 -0.27 -0.35 0.04 -0.26 -0.29 -0.07 -0.49 -0.24 1.5 -0.27 -0.04 -0.42 -0.09 -0.30 -0.38 0.15 -0.24 -0.32 -0.21 -0.49 -0.37 1.75 -0.16 -0.14 -0.38 -0.02 -0.36 -0.35 -0.03 -0.28 -0.29 -0.15 -0.56 -0.30 (1.5, 3) 1 -0.17 -0.18 -0.43 -0.19 -0.34 -0.35 0.08 -0.44 -0.37 -0.13 -0.49 -0.32 1.33 -0.16 -0.24 -0.35 -0.17 -0.41 -0.46 -0.04 -0.38 -0.49 -0.19 -0.56 -0.36 1.5 -0.07 -0.26 -0.53 -0.21 -0.45 -0.49 0.08 -0.40 -0.39 -0.17 -0.51 -0.33 1.75 -0.28 -0.25 -0.49 -0.18 -0.45 -0.45 0.00 -0.36 -0.40 -0.27 -0.45 -0.30 (1.75, 5) 1 -0.15 -0.36 -0.45 -0.09 -0.42 -0.44 0.01 -0.41 -0.37 -0.15 -0.64 -0.33 1.33 -0.18 -0.31 -0.47 -0.12 -0.38 -0.49 0.08 -0.46 -0.42 -0.27 -0.62 -0.38 1.5 -0.20 -0.32 -0.46 -0.22 -0.46 -0.55 0.05 -0.36 -0.45 -0.19 -0.57 -0.41 1.75 -0.13 -0.29 -0.47 -0.22 -0.36 -0.56 0.10 -0.42 -0.36 -0.20 -0.60 -0.46    139  Table 24. (cont™d) Number  of Indicators Level-1  Covariate Effect Level-2  Covariate Effect Number of Groups = 50 Number of Groups = 150 Group Size = 30 Group Size = 60 Group Size = 30 Group Size = 60 Quality of Indicators Quality of Indicators Quality of Indicators Quality of Indicators    0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 12 (1, 1) 1 -0.30 -0.29 -0.01 -0.12 -0.28 -0.07 -0.16 -0.31 0.00 -0.25 -0.32 0.00 1.33 -0.26 -0.26 0.00 -0.08 -0.28 -0.06 -0.18 -0.26 0.00 -0.27 -0.26 0.00 1.5 -0.20 -0.31 0.00 -0.17 -0.19 -0.08 -0.16 -0.24 0.00 -0.26 -0.28 0.00 1.75 -0.16 -0.33 0.01 -0.17 -0.31 -0.05 -0.21 -0.22 0.00 -0.27 -0.31 0.00 (1.33, 2) 1 -0.09 -0.21 -0.04 -0.15 -0.25 -0.06 -0.10 -0.27 -0.02 -0.19 -0.22 0.00 1.33 -0.05 -0.20 -0.08 -0.04 -0.23 0.00 -0.15 -0.29 -0.01 -0.20 -0.22 0.00 1.5 0.00 -0.25 -0.07 -0.03 -0.23 -0.01 -0.12 -0.25 -0.01 -0.19 -0.23 0.00 1.75 -0.03 -0.22 -0.05 -0.04 -0.24 -0.01 -0.15 -0.24 0.00 -0.15 -0.24 0.00 (1.5, 3) 1 -0.08 -0.27 -0.06 -0.09 -0.13 -0.02 -0.15 -0.21 0.00 -0.13 -0.19 0.00 1.33 -0.07 -0.26 -0.03 -0.06 -0.20 -0.01 -0.12 -0.12 0.00 -0.13 -0.15 0.00 1.5 -0.03 -0.28 -0.05 -0.08 -0.20 0.00 -0.15 -0.20 0.00 -0.13 -0.16 0.00 1.75 -0.09 -0.27 -0.05 -0.04 -0.23 -0.01 -0.10 -0.18 -0.01 -0.16 -0.12 0.00 (1.75, 5) 1 -0.08 -0.33 -0.18 -0.11 -0.17 -0.06 -0.09 -0.14 -0.04 -0.18 -0.10 -0.04 1.33 -0.11 -0.21 -0.14 -0.05 -0.18 -0.04 -0.18 -0.21 0.00 -0.21 -0.15 -0.06 1.5 -0.12 -0.20 -0.16 -0.04 -0.18 -0.04 -0.14 -0.24 -0.03 -0.16 -0.18 -0.05 1.75 -0.12 -0.22 -0.10 -0.01 -0.18 -0.06 -0.13 -0.20 -0.06 -0.19 -0.16 -0.05    140  Table 25. Bias of Conditional Response Probability of Latent Class 2 by Study Factors Number  of Indicators Level-1  Covariate Effect Level-2  Covariate Effect Number of Groups = 50 Number of Groups = 150 Group Size = 30 Group Size = 60 Group Size = 30 Group Size = 60 Quality of Indicators Quality of Indicators Quality of Indicators Quality of Indicators    0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 6 (1, 1) 1 0.24 0.12 0.08 0.33 0.14 0.06 0.27 0.20 0.12 0.56 0.19 0.09 1.33 0.32 0.17 0.07 0.54 0.17 0.12 0.31 0.26 0.13 0.48 0.14 0.04 1.5 0.26 0.18 0.15 0.58 0.19 0.09 0.32 0.22 0.16 0.44 0.14 0.06 1.75 0.38 0.22 0.06 0.50 0.13 0.10 0.20 0.21 0.15 0.49 0.13 0.11 (1.33, 2) 1 0.09 0.08 0.14 0.13 0.05 0.15 0.07 0.19 0.12 0.23 0.16 0.10 1.33 0.30 0.10 0.15 0.06 0.08 0.14 0.26 0.20 0.16 0.31 0.14 0.11 1.5 0.20 0.16 0.16 0.15 0.05 0.16 0.12 0.18 0.17 0.37 0.19 0.20 1.75 0.16 0.11 0.17 0.19 0.05 0.14 0.29 0.26 0.15 0.40 0.20 0.16 (1.5, 3) 1 0.12 0.18 0.16 0.21 0.12 0.18 0.11 0.10 0.17 0.07 0.13 0.17 1.33 0.28 0.16 0.12 0.34 0.14 0.22 0.17 0.13 0.26 0.13 0.19 0.16 1.5 0.23 0.16 0.19 0.19 0.13 0.23 0.05 0.15 0.22 0.09 0.16 0.13 1.75 0.28 0.13 0.11 0.08 0.08 0.21 0.01 0.14 0.23 0.11 0.18 0.14 (1.75, 5) 1 0.15 0.19 0.21 0.05 0.12 0.21 0.05 0.07 0.22 0.04 0.22 0.20 1.33 0.08 0.22 0.25 0.10 0.12 0.24 0.12 0.10 0.21 0.07 0.15 0.23 1.5 0.16 0.24 0.15 -0.05 0.12 0.26 0.04 0.09 0.24 0.07 0.15 0.24 1.75 0.22 0.19 0.19 -0.02 0.10 0.29 0.18 0.13 0.20 0.03 0.17 0.28    141  Table 25. (cont™d) Number  of Indicators Level-1  Covariate Effect Level-2  Covariate Effect Number of Groups = 50 Number of Groups = 150 Group Size = 30 Group Size = 60 Group Size = 30 Group Size = 60 Quality of Indicators Quality of Indicators Quality of Indicators Quality of Indicators    0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 12 (1, 1) 1 0.03 0.07 0.01 0.15 0.03 0.01 0.03 0.02 0.01 0.02 0.03 0.00 1.33 0.17 0.06 0.01 0.21 -0.01 0.02 0.10 0.02 0.01 0.03 0.04 0.00 1.5 0.08 0.12 0.01 0.12 -0.01 0.01 0.13 0.02 0.01 0.07 0.03 0.00 1.75 0.05 0.12 0.01 0.12 0.03 0.01 0.08 0.04 0.01 0.04 0.01 0.00 (1.33, 2) 1 0.16 0.05 0.00 0.18 0.02 0.03 0.04 0.03 0.02 0.08 0.02 0.00 1.33 0.03 0.10 0.00 0.30 0.03 0.01 0.13 0.03 0.01 0.09 0.01 0.00 1.5 -0.06 0.08 -0.01 0.18 0.04 0.01 0.14 0.03 0.01 0.10 0.01 0.00 1.75 -0.09 0.06 -0.01 0.14 0.01 0.01 0.02 0.03 0.00 0.07 0.02 0.00 (1.5, 3) 1 0.01 0.03 -0.01 0.12 0.03 0.00 0.08 0.03 0.00 0.07 0.01 0.00 1.33 -0.05 0.05 -0.03 0.18 0.02 0.00 0.04 0.02 0.00 0.06 0.01 0.00 1.5 0.07 0.00 -0.03 0.19 0.02 0.00 0.10 0.01 0.00 0.06 0.01 0.00 1.75 -0.05 0.01 -0.03 0.03 0.02 0.01 0.07 0.01 -0.01 0.03 0.01 0.00 (1.75, 5) 1 0.08 0.03 -0.07 0.06 0.01 -0.02 -0.03 0.00 -0.02 0.08 0.00 -0.04 1.33 0.16 0.01 -0.05 0.04 0.02 0.01 0.01 0.01 0.00 0.11 0.00 -0.04 1.5 0.22 0.01 -0.06 0.07 0.01 -0.01 -0.01 0.00 -0.01 0.07 0.00 -0.05 1.75 0.14 0.00 -0.04 0.09 0.00 -0.03 -0.01 -0.01 -0.06 0.10 0.00 -0.05    142  Table 26. Bias of Conditional Response Probability of Latent Class 3 by Study Factors Number  of Indicators Level-1  Covariate Effect Level-2  Covariate Effect Number of Groups = 50 Number of Groups = 150 Group Size = 30 Group Size = 60 Group Size = 30 Group Size = 60 Quality of Indicators Quality of Indicators Quality of Indicators Quality of Indicators    0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 6 (1, 1) 1 0.23 -0.15 -0.01 0.37 0.14 0.27 0.26 0.07 0.13 0.29 0.29 0.21 1.33 0.39 -0.13 -0.02 0.18 0.06 0.27 0.48 0.22 0.11 0.25 0.29 0.14 1.5 0.28 -0.22 0.04 0.24 0.07 0.29 0.21 0.19 0.13 0.27 0.24 0.18 1.75 0.13 -0.30 0.06 0.35 -0.01 0.19 0.30 0.09 0.12 0.34 0.17 0.18 (1.33, 2) 1 0.38 0.04 0.04 0.40 0.16 0.20 0.25 0.03 0.11 0.17 0.23 0.10 1.33 0.24 -0.24 0.14 0.50 0.09 0.17 0.36 0.07 0.11 0.20 0.27 0.12 1.5 0.34 -0.27 0.11 0.44 0.18 0.18 0.18 0.10 0.13 0.24 0.24 0.16 1.75 0.41 -0.30 0.04 0.39 0.21 0.16 0.27 0.08 0.12 0.24 0.30 0.13 (1.5, 3) 1 0.44 0.02 0.18 0.08 0.20 0.13 0.17 0.31 0.18 0.17 0.32 0.14 1.33 0.21 0.24 0.12 0.09 0.23 0.20 0.30 0.26 0.21 0.17 0.33 0.18 1.5 0.23 0.11 0.23 0.10 0.23 0.21 0.48 0.26 0.15 0.13 0.31 0.19 1.75 0.41 0.15 0.28 0.18 0.26 0.20 0.36 0.24 0.16 0.17 0.23 0.15 (1.75, 5) 1 0.28 0.15 0.18 0.05 0.23 0.20 0.22 0.33 0.14 0.21 0.39 0.12 1.33 0.35 0.14 0.14 0.26 0.18 0.21 0.14 0.34 0.20 0.21 0.43 0.15 1.5 0.43 0.10 0.24 0.35 0.23 0.24 0.26 0.24 0.20 0.15 0.38 0.16 1.75 0.31 0.02 0.20 0.27 0.15 0.22 0.14 0.28 0.15 0.27 0.40 0.17    143  Table 26. (cont™d) Number  of Indicators Level-1  Covariate Effect Level-2  Covariate Effect Number of Groups = 50 Number of Groups = 150 Group Size = 30 Group Size = 60 Group Size = 30 Group Size = 60 Quality of Indicators Quality of Indicators Quality of Indicators Quality of Indicators    0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 12 (1, 1) 1 0.29 0.31 0.01 0.22 0.29 0.07 0.19 0.30 0.00 0.21 0.29 0.00 1.33 0.32 0.26 -0.01 0.22 0.32 0.05 0.17 0.25 0.00 0.20 0.23 0.00 1.5 0.45 0.26 -0.01 0.29 0.23 0.09 0.18 0.24 0.00 0.17 0.26 0.00 1.75 0.22 0.24 -0.01 0.26 0.32 0.05 0.18 0.20 0.00 0.14 0.30 0.00 (1.33, 2) 1 0.14 0.20 0.03 0.19 0.27 0.04 0.09 0.25 -0.01 0.18 0.21 0.00 1.33 0.17 0.15 0.07 0.04 0.23 0.00 0.10 0.26 -0.01 0.18 0.22 0.00 1.5 0.14 0.24 0.07 -0.07 0.23 0.00 0.05 0.21 -0.01 0.16 0.22 0.00 1.75 0.10 0.23 0.06 0.12 0.26 0.02 0.21 0.20 -0.01 0.19 0.23 0.00 (1.5, 3) 1 0.23 0.28 0.05 0.06 0.13 0.02 0.13 0.18 -0.01 0.06 0.19 0.00 1.33 0.23 0.23 0.05 0.06 0.21 0.02 0.17 0.10 -0.01 0.05 0.14 0.00 1.5 0.10 0.27 0.07 0.04 0.21 0.00 0.21 0.19 -0.01 0.04 0.15 0.00 1.75 0.12 0.25 0.07 0.15 0.23 0.00 0.22 0.18 0.01 0.11 0.11 0.00 (1.75, 5) 1 0.17 0.29 0.24 0.11 0.18 0.08 0.13 0.14 0.06 0.11 0.10 0.08 1.33 0.10 0.18 0.18 0.11 0.17 0.03 0.15 0.20 -0.01 0.14 0.16 0.10 1.5 0.17 0.17 0.21 0.07 0.19 0.05 0.13 0.24 0.05 0.12 0.19 0.10 1.75 0.18 0.20 0.13 -0.02 0.20 0.10 0.13 0.21 0.11 0.13 0.15 0.10    144  Table 27. Standard Error Biases of Level-1 Covariate Effect on the Latent Class 1 by Study Factors Number  of Indicators Level-1  Covariate Effect Level-2  Covariate Effect Number of Groups = 50 Number of Groups = 150 Group Size = 30 Group Size = 60 Group Size = 30 Group Size = 60 Quality of Indicators Quality of Indicators Quality of Indicators Quality of Indicators    0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 6 (1, 1) 1 -0.67 -0.96 0.05 9.13 -0.09 0.04 0.88 0.07 0.04 -0.54 0.16 -0.09 1.33 -0.77 -0.72 0.12 -1.00 -0.41 0.03 -0.63 -0.04 0.06 -0.75 0.17 -0.14 1.5 -0.76 -0.55 0.19 -0.18 -0.39 0.05 -0.67 0.02 0.10 -0.46 0.21 -0.08 1.75 -0.66 -0.48 0.08 -0.37 57.78 0.05 -0.64 0.10 0.13 -0.94 0.10 -0.08 (1.33, 2) 1 -0.87 -0.98 -0.27 -0.99 -0.80 -0.53 -0.61 -0.33 -0.54 -0.81 -0.47 -0.65 1.33 8.01 -0.70 -0.26 -0.73 -0.29 -0.51 -1.00 0.50 -0.59 -0.69 -0.45 -0.68 1.5 -0.91 -0.71 -0.32 -1.00 -0.25 -0.52 -1.00 -0.02 -0.60 -0.87 -0.47 -0.73 1.75 -0.92 -0.55 -0.35 -1.00 -0.26 -0.51 -1.00 -0.24 -0.58 -0.88 -0.50 -0.71 (1.5, 3) 1 -1.00 -0.62 -0.49 -1.00 -0.66 -0.63 -1.00 -0.17 -0.69 -0.29 -0.57 -0.77 1.33 -1.00 -0.45 -0.43 -1.00 -0.41 -0.65 -0.88 0.09 -0.73 -0.23 -0.57 -0.78 1.5 -1.00 -0.49 -0.50 -1.00 -0.36 -0.66 -0.93 -0.10 -0.70 1.50 -0.55 -0.77 1.75 -1.00 -0.62 -0.48 -0.95 -0.24 -0.65 -0.67 0.17 -0.70 10.52 -0.56 -0.76 (1.75, 5) 1 -0.77 0.60 -0.52 -1.00 4.10 -0.70 -1.00 -0.26 -0.74 0.18 -0.70 -0.81 1.33 -0.99 -0.62 -0.51 -1.00 -0.38 -0.70 -0.99 -0.70 -0.75 -0.99 -0.67 -0.82 1.5 -0.93 -0.71 -0.53 -1.00 -0.36 -0.72 -1.00 -0.53 -0.75 -0.49 -0.69 -0.82 1.75 -0.98 -0.71 -0.46 -0.98 -0.47 -0.72 -0.99 -0.96 -0.73 -0.73 -0.71 -0.83    145  Table 27. (cont™d) Number  of Indicators Level-1  Covariate Effect Level-2  Covariate Effect Number of Groups = 50 Number of Groups = 150 Group Size = 30 Group Size = 60 Group Size = 30 Group Size = 60 Quality of Indicators Quality of Indicators Quality of Indicators Quality of Indicators    0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 12 (1, 1) 1 -0.98 0.55 -0.10 -0.92 0.18 0.08 0.25 -0.04 -0.04 -0.56 0.08 0.14 1.33 -0.96 0.40 -0.09 -0.94 0.17 0.06 -0.52 0.09 -0.03 -0.63 0.09 0.14 1.5 4.39 0.27 -0.09 -0.94 0.18 0.05 -0.51 0.02 -0.04 -0.41 0.08 0.13 1.75 4.18 0.13 -0.10 -0.95 0.19 0.06 -0.51 -0.03 -0.04 -0.67 0.09 0.13 (1.33, 2) 1 51.78 -0.21 -0.14 -0.62 -0.23 -0.10 -0.29 -0.43 -0.14 -0.40 -0.50 0.18 1.33 -0.92 -0.30 -0.19 -0.69 -0.25 0.04 -0.41 -0.45 -0.06 -0.05 -0.46 0.18 1.5 -0.99 -0.22 -0.20 -0.66 -0.26 0.04 -0.42 -0.42 -0.07 -0.35 -0.48 0.16 1.75 -1.00 0.10 -0.17 -0.77 -0.24 -0.04 -0.23 -0.37 -0.02 -0.90 -0.51 0.16 (1.5, 3) 1 -0.99 0.06 -0.21 -1.00 -0.20 -0.09 -0.48 -0.45 -0.03 15.56 -0.54 0.15 1.33 -0.56 -0.18 -0.11 -0.38 -0.32 0.03 -0.70 -0.39 -0.01 -0.21 -0.49 0.16 1.5 -0.98 -0.31 -0.16 -0.91 -0.26 0.04 -0.53 -0.44 -0.03 0.30 -0.48 0.15 1.75 -0.84 -0.21 -0.15 -0.59 -0.22 -0.01 -0.44 -0.42 -0.03 -0.18 -0.41 0.15 (1.75, 5) 1 -0.67 -0.40 -0.42 -0.95 -0.18 -0.01 -0.40 -0.38 -0.31 0.62 -0.44 -0.30 1.33 -0.89 -0.61 -0.34 -0.36 -0.23 -0.23 -0.63 -0.48 0.01 -0.20 -0.52 -0.45 1.5 -0.88 -0.32 -0.39 -0.81 -0.21 -0.19 -0.11 -0.48 -0.26 -0.44 -0.54 -0.34 1.75 -0.98 -0.37 -0.27 -0.96 -0.20 -0.26 0.38 -0.45 -0.19 -0.44 -0.50 -0.32    146  Table 28. Standard Error Biases of Level-1 Covariate Effect on the Latent Class 2 by Study Factors Number  of Indicators Level-1  Covariate Effect Level-2  Covariate Effect Number of Groups = 50 Number of Groups = 150 Group Size = 30 Group Size = 60 Group Size = 30 Group Size = 60 Quality of Indicators Quality of Indicators Quality of Indicators Quality of Indicators    0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 6 (1, 1) 1 -0.98 -0.99 0.08 2.20 -0.89 0.20 -0.66 0.18 0.19 -0.99 0.04 0.02 1.33 -1.00 -0.97 0.06 -1.00 -0.80 0.17 -1.00 0.41 0.17 -0.73 0.06 0.05 1.5 -0.99 -0.97 0.10 -0.85 -0.79 0.09 -0.98 -0.89 0.14 -0.94 0.03 0.06 1.75 -1.00 -0.97 0.06 -0.99 15.31 0.09 -0.99 -0.78 0.14 -0.95 0.05 0.10 (1.33, 2) 1 -0.99 -0.43 -0.35 -0.99 -0.23 -0.53 -0.52 -0.33 -0.58 -0.90 -0.54 -0.66 1.33 -0.71 -0.48 -0.41 -0.88 -0.23 -0.52 -0.98 0.27 -0.59 -0.94 -0.53 -0.69 1.5 -1.00 -0.52 -0.37 -0.93 -0.23 -0.49 -0.98 -0.13 -0.60 -1.00 -0.53 -0.73 1.75 -1.00 -0.97 -0.36 -1.00 -0.98 -0.47 -1.00 -0.22 -0.60 -1.00 -0.56 -0.72 (1.5, 3) 1 -1.00 -0.72 -0.55 -0.99 -0.39 -0.67 -1.00 -0.42 -0.74 -1.00 -0.67 -0.81 1.33 -1.00 -0.62 -0.51 -1.00 -0.44 -0.70 -0.99 -0.39 -0.76 -0.51 -0.53 -0.82 1.5 -1.00 -0.70 -0.55 -1.00 -0.35 -0.70 -0.99 -0.42 -0.73 0.69 -0.64 -0.81 1.75 -1.00 -0.89 -0.56 -0.95 -0.30 -0.69 -0.99 -0.12 -0.74 18.57 -0.64 -0.80 (1.75, 5) 1 -0.80 0.39 -0.60 -1.00 3.71 -0.74 -1.00 -0.67 -0.79 0.02 -0.77 -0.84 1.33 -0.99 -0.60 -0.59 -1.00 -0.07 -0.75 -0.99 -0.53 -0.80 -0.99 -0.77 -0.85 1.5 -0.92 -0.49 -0.62 -1.00 -0.08 -0.76 -0.94 -0.60 -0.81 -0.08 -0.77 -0.85 1.75 -0.97 -0.62 -0.56 -0.98 -0.31 -0.76 -0.98 -0.97 -0.79 -0.21 -0.74 -0.86    147  Table 28. (cont™d) Number  of Indicators Level-1  Covariate Effect Level-2  Covariate Effect Number of Groups = 50 Number of Groups = 150 Group Size = 30 Group Size = 60 Group Size = 30 Group Size = 60 Quality of Indicators Quality of Indicators Quality of Indicators Quality of Indicators    0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 12 (1, 1) 1 -0.93 0.03 -0.10 -0.91 0.03 0.09 1.37 0.07 -0.05 -0.75 0.08 0.03 1.33 -0.72 0.02 -0.09 -0.91 0.07 0.12 -0.33 0.12 -0.06 -0.64 0.09 0.04 1.5 0.16 -0.17 -0.09 -0.93 0.09 0.10 -0.54 0.06 -0.06 -0.52 0.09 0.03 1.75 -0.97 -0.11 -0.09 -0.93 0.07 0.12 -0.46 0.08 -0.05 -0.69 0.07 0.04 (1.33, 2) 1 72.01 -0.30 -0.19 -0.66 -0.21 -0.08 1.29 -0.33 -0.09 -0.07 -0.29 0.03 1.33 -0.92 -0.24 -0.20 -0.66 -0.23 0.03 -0.68 -0.09 -0.09 -0.14 -0.18 0.04 1.5 -0.99 -0.25 -0.24 -0.62 -0.19 0.00 -0.50 -0.23 -0.09 -0.28 -0.28 0.02 1.75 -1.00 0.06 -0.25 -0.52 -0.21 0.01 -0.24 -0.22 -0.05 -0.91 -0.27 0.02 (1.5, 3) 1 -0.98 0.05 -0.38 -1.00 -0.16 -0.03 -0.07 -0.29 -0.05 9.91 -0.27 0.00 1.33 -0.99 -0.14 -0.43 -0.61 -0.26 -0.29 -0.73 -0.33 -0.04 -0.30 -0.27 -0.01 1.5 -0.96 -0.29 -0.45 -1.00 -0.29 -0.03 -0.35 -0.32 -0.06 2.66 -0.26 -0.02 1.75 -1.00 -0.26 -0.44 -0.63 -0.31 -0.11 -0.01 -0.39 -0.39 -0.41 -0.19 -0.01 (1.75, 5) 1 -0.65 -0.26 -0.64 -0.74 -0.04 -0.57 -0.35 -0.11 -0.67 0.45 -0.17 -0.78 1.33 -0.87 -0.63 -0.58 0.01 -0.05 -0.42 -0.40 -0.35 -0.01 0.18 -0.48 -0.78 1.5 -0.85 -0.25 -0.61 -0.98 -0.18 -0.56 -0.94 -0.34 -0.55 -0.31 -0.49 -0.80 1.75 -0.92 -0.40 -0.56 -0.94 -0.25 -0.67 -0.16 -0.43 -0.74 -0.41 -0.29 -0.80    148  Table 29. Standard Error Biases of Level-2 Covariate Effect by Study Factors Number  of Indicators Level-1  Covariate Effect Level-2  Covariate Effect Number of Groups = 50 Number of Groups = 150 Group Size = 30 Group Size = 60 Group Size = 30 Group Size = 60 Quality of Indicators Quality of Indicators Quality of Indicators Quality of Indicators    0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 6 (1, 1) 1 -1.00 -0.82 -0.70 -0.93 -0.79 -0.72 -1.00 0.16 0.02 -0.73 0.16 0.17 1.33 -1.00 -0.91 -1.00 -1.00 -0.70 -0.92 -1.00 -0.99 0.03 -0.76 0.15 0.20 1.5 -1.00 -1.00 -0.69 -0.99 -0.79 -0.84 -1.00 -0.99 0.03 -0.91 0.13 0.16 1.75 -1.00 -0.96 -0.73 -1.00 76.86 -0.74 -1.00 0.16 0.01 -0.63 0.13 0.16 (1.33, 2) 1 -1.00 -0.86 -0.66 -1.00 -0.61 -0.73 -1.00 -0.70 0.02 -0.79 0.15 0.16 1.33 -0.99 -0.86 -0.66 -1.00 -0.66 -0.72 -0.99 -0.53 0.06 -0.99 0.14 0.15 1.5 -0.99 -0.99 -0.91 -1.00 -0.69 -0.72 -1.00 -0.72 0.04 -0.49 0.12 0.13 1.75 -1.00 -0.88 -1.00 -1.00 -0.76 -0.73 -0.99 -0.74 0.03 -0.70 0.12 0.13 (1.5, 3) 1 -0.98 -1.00 -0.72 -0.98 -0.58 0.09 -1.00 -1.00 0.05 -1.00 0.06 0.09 1.33 -0.99 -1.00 -1.00 -0.96 -0.67 -1.00 -1.00 -0.97 0.07 -0.83 0.31 0.11 1.5 -0.99 -1.00 -0.76 -0.99 -0.67 -1.00 -1.00 -0.96 0.07 -0.57 0.14 0.09 1.75 -1.00 -0.97 -0.99 -0.99 0.08 -0.63 -1.00 -0.80 0.05 -0.73 0.07 0.08 (1.75, 5) 1 -1.00 -1.00 -0.74 -1.00 -0.51 0.13 -0.99 -1.00 0.06 -0.81 0.07 0.18 1.33 -1.00 -0.98 -0.79 -0.99 -0.74 -0.71 -1.00 -0.36 0.09 -1.00 0.09 0.17 1.5 -1.00 -0.98 -0.80 -0.65 -0.74 -0.71 -0.96 -1.00 0.10 -0.65 0.07 0.13 1.75 -0.99 -1.00 -0.75 -1.00 -0.74 -0.71 -0.98 -1.00 0.10 -0.85 0.10 0.13    149  Table 29. (cont™d) Number  of Indicators Level-1  Covariate Effect Level-2  Covariate Effect Number of Groups = 50 Number of Groups = 150 Group Size = 30 Group Size = 60 Group Size = 30 Group Size = 60 Quality of Indicators Quality of Indicators Quality of Indicators Quality of Indicators    0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 12 (1, 1) 1 -1.00 -0.99 -0.05 -0.97 0.08 -0.05 -1.00 0.05 0.10 -0.82 0.19 0.23 1.33 -0.91 -0.99 -0.29 -1.00 -0.68 -0.03 -1.00 0.08 0.08 -0.83 0.17 0.19 1.5 5.20 -0.54 -0.30 -1.00 -0.66 -0.05 -0.94 0.07 0.08 -0.79 0.12 0.18 1.75 -0.81 -0.57 -0.34 -1.00 0.12 -0.04 -1.00 0.10 0.05 -0.77 0.10 0.16 (1.33, 2) 1 -1.00 -0.76 -0.51 -1.00 -0.04 -0.69 -1.00 0.04 0.04 41.16 0.19 0.28 1.33 -1.00 -0.99 0.00 -1.00 0.03 -0.71 -1.00 0.07 0.03 -0.68 0.17 0.25 1.5 -1.00 -0.99 -0.01 -1.00 0.05 -0.74 -0.90 0.07 0.04 0.44 0.16 0.25 1.75 -1.00 -0.98 -0.02 -1.00 0.02 -0.74 -0.99 0.08 0.02 -0.72 0.13 0.22 (1.5, 3) 1 -0.99 -0.62 -0.07 -0.98 -0.06 -0.67 -0.98 0.07 0.07 -0.41 0.24 0.33 1.33 -0.99 -0.66 -0.07 -0.90 -0.05 -0.74 -0.97 0.07 0.06 -0.62 0.20 0.27 1.5 -0.99 -0.80 -0.08 -0.98 -0.03 -0.74 -0.91 0.09 0.06 3.59 0.18 0.26 1.75 -1.00 -0.85 -0.11 -1.00 -0.02 -0.73 -0.99 0.10 0.03 -0.66 0.16 0.23 (1.75, 5) 1 -0.94 0.08 -0.11 -0.89 -0.01 -0.14 -1.00 0.10 0.11 -1.00 0.17 0.33 1.33 -1.00 -0.78 -0.10 -0.90 0.03 -0.74 -1.00 0.06 0.07 -0.72 0.16 0.28 1.5 -1.00 -0.73 -0.10 -0.99 0.06 -0.61 -0.92 0.06 0.06 -0.73 0.15 0.27 1.75 -1.00 0.01 -0.10 -0.99 0.06 -0.73 -0.98 0.05 0.05 -0.71 0.16 0.27    150  Table 30. Standard Error Biases of Conditional Response Probability of Latent Class 1 by Study Factors Number  of Indicators Level-1  Covariate Effect Level-2  Covariate Effect Number of Groups = 50 Number of Groups = 150 Group Size = 30 Group Size = 60 Group Size = 30 Group Size = 60 Quality of Indicators Quality of Indicators Quality of Indicators Quality of Indicators    0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 6 (1, 1) 1 -0.34 -0.67 -0.67 -0.49 -0.57 -0.78 0.48 -0.63 -0.82 -0.53 -0.62 -0.88 1.33 -0.35 -0.65 -0.63 -0.54 -0.24 -0.79 -0.73 -0.70 -0.82 -0.47 -0.60 -0.86 1.5 -0.41 -0.64 -0.66 -0.22 -0.38 -0.79 0.08 -0.70 -0.83 -0.57 -0.58 -0.87 1.75 13.89 -0.61 -0.62 -0.37 66.37 -0.78 -0.58 -0.62 -0.83 8.06 -0.64 -0.88 (1.33, 2) 1 -0.67 -0.75 -0.63 -0.47 0.39 -0.78 -0.31 -0.47 -0.81 -0.49 -0.69 -0.87 1.33 12.54 0.38 -0.70 -0.75 -0.48 -0.77 -0.68 -0.25 -0.81 9.54 -0.77 -0.87 1.5 -0.75 -0.46 -0.68 -0.66 -0.56 -0.77 -0.47 -0.43 -0.82 -0.50 -0.66 -0.88 1.75 -0.78 -0.61 -0.69 -0.66 -0.59 -0.78 -0.54 -0.53 -0.81 -0.67 -0.64 -0.88 (1.5, 3) 1 -0.76 0.01 -0.68 -0.56 -0.42 -0.79 -0.59 -0.50 -0.83 0.57 -0.70 -0.88 1.33 -0.56 -0.11 -0.70 -0.68 -0.47 -0.80 -0.34 -0.30 -0.84 -0.39 -0.65 -0.88 1.5 -0.73 -0.32 -0.71 -0.29 -0.49 -0.79 -0.62 -0.47 -0.83 -0.11 -0.70 -0.88 1.75 -0.75 -0.31 -0.71 -0.65 -0.50 -0.79 -0.18 -0.44 -0.83 -0.34 -0.70 -0.88 (1.75, 5) 1 -0.66 -0.71 -0.72 -0.69 -0.33 -0.80 -0.44 -0.54 -0.83 -0.37 -0.70 -0.88 1.33 -0.69 -0.64 -0.69 -0.25 -0.47 -0.80 -0.33 -0.51 -0.84 -0.32 -0.70 -0.89 1.5 88.64 -0.64 -0.70 -0.14 -0.46 -0.80 -0.16 -0.55 -0.84 -0.40 -0.72 -0.89 1.75 -0.65 0.45 -0.73 -0.43 -0.50 -0.80 -0.32 -0.57 -0.83 -0.46 -0.71 -0.89    151  Table 30. (cont™d) Number  of Indicators Level-1  Covariate Effect Level-2  Covariate Effect Number of Groups = 50 Number of Groups = 150 Group Size = 30 Group Size = 60 Group Size = 30 Group Size = 60 Quality of Indicators Quality of Indicators Quality of Indicators Quality of Indicators    0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 12 (1, 1) 1 -0.73 -0.35 -0.45 -0.37 -0.71 -0.76 -0.46 -0.77 0.01 -0.62 -0.86 0.01 1.33 -0.36 -0.43 0.03 -0.62 -0.70 -0.75 -0.59 -0.69 0.00 -0.64 -0.85 0.01 1.5 14.56 -0.48 0.03 -0.62 -0.68 -0.79 -0.45 -0.75 0.00 -0.10 -0.85 0.01 1.75 18.65 -0.50 0.02 -0.67 -0.71 -0.73 -0.50 -0.72 0.00 -0.60 -0.85 0.01 (1.33, 2) 1 -0.27 -0.48 -0.59 -0.66 -0.70 -0.74 -0.29 -0.76 -0.56 0.52 -0.84 0.01 1.33 -0.66 -0.52 -0.68 -0.65 -0.70 0.01 -0.57 -0.77 -0.45 -0.25 -0.84 0.01 1.5 65.82 -0.53 -0.67 -0.71 -0.70 -0.39 -0.51 -0.76 -0.45 -0.32 -0.84 0.01 1.75 -0.22 -0.48 -0.63 -0.61 -0.71 -0.56 -0.56 -0.76 0.01 -0.57 -0.84 0.01 (1.5, 3) 1 -0.66 -0.60 -0.63 -0.52 -0.67 -0.56 -0.53 -0.76 0.01 0.65 -0.83 0.01 1.33 -0.16 -0.60 -0.48 -0.60 -0.70 -0.38 -0.66 -0.71 0.01 -0.40 -0.82 0.01 1.5 0.09 -0.63 -0.59 31.00 -0.70 0.00 -0.55 -0.75 0.01 3.22 -0.82 0.01 1.75 0.98 -0.57 -0.59 -0.56 -0.71 -0.37 -0.47 -0.75 -0.46 -0.38 -0.80 0.01 (1.75, 5) 1 -0.58 -0.59 -0.76 -0.52 -0.69 -0.64 -0.52 -0.73 -0.67 -0.24 -0.79 -0.76 1.33 -0.70 -0.51 -0.73 0.02 -0.69 -0.67 -0.59 -0.75 0.01 -0.17 -0.82 -0.83 1.5 -0.67 -0.55 -0.75 -0.61 -0.69 -0.62 -0.42 -0.76 -0.71 -0.28 -0.83 -0.78 1.75 -0.75 -0.55 -0.69 -0.58 -0.69 -0.69 -0.39 -0.75 -0.72 -0.17 -0.82 -0.78    152  Table 31. Standard Error Biases of Conditional Response Probability of Latent Class 2 by Study Factors Number  of Indicators Level-1  Covariate Effect Level-2  Covariate Effect Number of Groups = 50 Number of Groups = 150 Group Size = 30 Group Size = 60 Group Size = 30 Group Size = 60 Quality of Indicators Quality of Indicators Quality of Indicators Quality of Indicators    0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 6 (1, 1) 1 -0.65 -0.50 -0.34 -0.63 -0.52 -0.61 -0.32 -0.43 -0.60 -0.03 -0.60 -0.78 1.33 -0.80 -0.72 -0.26 -0.48 -0.32 -0.62 -0.57 -0.51 -0.58 0.88 -0.45 -0.72 1.5 -0.70 -0.66 -0.27 -0.32 -0.48 -0.62 -0.66 -0.47 -0.60 0.68 -0.53 -0.76 1.75 -0.67 -0.54 -0.39 0.32 29.09 -0.56 -0.72 -0.40 -0.60 -0.02 -0.47 -0.76 (1.33, 2) 1 -0.68 -0.70 -0.50 -0.70 -0.30 -0.65 -0.45 -0.68 -0.66 -0.66 -0.55 -0.76 1.33 0.23 -0.68 -0.55 -0.67 -0.23 -0.65 -0.79 -0.40 -0.65 -0.01 -0.56 -0.76 1.5 -0.64 -0.66 -0.52 -0.57 -0.51 -0.65 -0.48 -0.59 -0.65 -0.53 -0.55 -0.77 1.75 -0.66 -0.69 -0.50 -0.63 -0.34 -0.60 -0.70 -0.65 -0.63 -0.44 -0.58 -0.76 (1.5, 3) 1 -0.59 -0.70 -0.56 -0.61 -0.56 -0.63 -0.68 -0.43 -0.75 8.00 -0.67 -0.82 1.33 -0.23 -0.66 -0.55 -0.23 -0.60 -0.70 -0.77 -0.67 -0.75 -0.52 -0.41 -0.84 1.5 -0.67 -0.67 -0.59 0.13 -0.64 -0.71 -0.71 -0.67 -0.71 0.76 -0.65 -0.84 1.75 -0.65 -0.66 -0.61 -0.71 -0.57 -0.70 -0.36 -0.67 -0.72 -0.14 -0.65 -0.83 (1.75, 5) 1 -0.72 -0.70 -0.63 -0.57 -0.26 -0.75 -0.53 -0.52 -0.75 0.26 -0.72 -0.84 1.33 -0.77 -0.72 -0.61 -0.60 -0.54 -0.75 -0.67 -0.44 -0.80 -0.39 -0.71 -0.83 1.5 33.39 -0.75 -0.66 -0.47 -0.61 -0.76 -0.40 -0.42 -0.76 -0.20 -0.73 -0.86 1.75 -0.72 -0.67 -0.65 -0.38 -0.55 -0.74 -0.48 -0.41 -0.75 0.49 -0.73 -0.86    153  Table 31. (cont™d) Number  of Indicators Level-1  Covariate Effect Level-2  Covariate Effect Number of Groups = 50 Number of Groups = 150 Group Size = 30 Group Size = 60 Group Size = 30 Group Size = 60 Quality of Indicators Quality of Indicators Quality of Indicators Quality of Indicators    0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 12 (1, 1) 1 -0.63 -0.32 -0.02 -0.43 -0.15 0.05 40.08 -0.28 0.03 -0.44 -0.53 0.01 1.33 -0.49 -0.40 -0.01 -0.49 -0.19 -0.27 -0.42 -0.14 0.04 -0.35 -0.59 0.01 1.5 0.96 -0.49 -0.02 -0.35 -0.13 0.04 -0.51 -0.29 0.04 -0.23 -0.56 0.00 1.75 1.71 -0.43 -0.02 -0.16 -0.22 0.06 -0.37 -0.38 0.03 -0.47 -0.63 0.01 (1.33, 2) 1 -0.55 -0.46 -0.25 -0.66 -0.14 -0.51 2.19 -0.41 -0.51 12.39 -0.30 0.02 1.33 -0.60 -0.39 -0.25 7.04 -0.26 0.04 -0.38 -0.29 -0.39 -0.30 0.07 0.02 1.5 3.22 -0.46 -0.37 -0.65 -0.16 -0.33 -0.61 -0.38 -0.39 -0.43 -0.18 0.02 1.75 -0.64 -0.37 -0.39 -0.61 -0.17 0.04 -0.42 -0.44 0.04 -0.32 -0.30 0.02 (1.5, 3) 1 -0.68 -0.60 -0.49 -0.72 -0.12 0.05 2.64 -0.31 0.02 1.86 0.06 0.02 1.33 -0.63 -0.61 -0.48 -0.59 -0.33 -0.37 -0.54 -0.36 0.03 -0.27 -0.23 0.02 1.5 -0.39 -0.49 -0.48 1.61 -0.29 0.03 -0.51 -0.28 0.02 3.66 0.06 0.02 1.75 -0.45 -0.58 -0.48 -0.61 -0.22 -0.34 0.01 -0.33 -0.43 -0.31 0.06 0.02 (1.75, 5) 1 -0.69 -0.25 -0.69 -0.67 0.10 -0.62 -0.49 0.06 -0.74 -0.32 0.05 -0.77 1.33 -0.72 -0.15 -0.64 -0.52 -0.07 -0.62 -0.38 -0.36 0.03 -0.25 -0.34 -0.77 1.5 -0.63 -0.22 -0.65 -0.70 -0.23 -0.69 -0.29 -0.13 -0.55 -0.21 -0.34 -0.79 1.75 -0.71 -0.31 -0.66 -0.61 -0.20 -0.74 -0.35 -0.21 -0.72 -0.33 0.05 -0.79    154  Table 32. Standard Error Biases of Conditional Response Probability of Latent Class 3 by Study Factors Number  of Indicators Level-1  Covariate Effect Level-2  Covariate Effect Number of Groups = 50 Number of Groups = 150 Group Size = 30 Group Size = 60 Group Size = 30 Group Size = 60 Quality of Indicators Quality of Indicators Quality of Indicators Quality of Indicators    0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 6 (1, 1) 1 -0.34 -0.59 -0.35 1.66 -0.29 -0.59 1.78 -0.37 -0.57 -0.47 -0.35 -0.78 1.33 -0.65 -0.58 -0.32 0.48 -0.37 -0.59 -0.59 -0.42 -0.52 0.69 -0.41 -0.74 1.5 -0.50 -0.46 1.01 0.58 -0.31 -0.63 -0.69 -0.44 -0.54 -0.45 -0.41 -0.77 1.75 -0.25 -0.36 -0.39 -0.08 18.47 -0.58 -0.71 -0.32 -0.54 1.72 -0.30 -0.75 (1.33, 2) 1 -0.61 -0.61 -0.49 -0.04 0.28 -0.59 -0.46 -0.43 -0.56 -0.06 -0.50 -0.66 1.33 62.93 -0.63 -0.50 0.41 -0.15 -0.57 -0.57 0.24 -0.52 4.52 -0.54 -0.68 1.5 -0.56 -0.35 -0.41 -0.49 -0.48 -0.57 -0.64 -0.27 -0.53 -0.23 -0.49 -0.68 1.75 -0.65 -0.66 -0.46 1.07 -0.40 -0.49 -0.16 -0.34 -0.51 -0.33 -0.53 -0.65 (1.5, 3) 1 -0.39 -0.58 -0.53 -0.26 0.20 -0.44 -0.47 -0.26 -0.64 4.21 -0.64 -0.71 1.33 -0.67 -0.61 -0.51 -0.52 -0.41 -0.54 0.31 -0.08 -0.63 0.08 -0.61 -0.76 1.5 -0.67 -0.66 -0.56 -0.29 -0.42 -0.55 -0.28 -0.13 -0.56 6.58 -0.60 -0.77 1.75 -0.70 -0.53 -0.57 -0.55 -0.45 -0.53 -0.03 -0.20 -0.57 1.94 -0.55 -0.74 (1.75, 5) 1 -0.66 -0.53 -0.49 -0.68 2.94 -0.60 -0.54 -0.46 -0.58 0.73 -0.71 -0.71 1.33 -0.62 -0.61 -0.44 -0.47 -0.30 -0.59 -0.50 -0.47 -0.66 -0.31 -0.72 -0.69 1.5 12.25 -0.53 -0.49 -0.05 -0.42 -0.61 2.77 -0.26 -0.62 -0.14 -0.70 -0.73 1.75 -0.71 0.35 -0.40 -0.45 -0.43 -0.59 -0.38 -0.45 -0.58 1.22 -0.70 -0.74    155  Table 32. (cont™d) Number  of Indicators Level-1  Covariate Effect Level-2  Covariate Effect Number of Groups = 50 Number of Groups = 150 Group Size = 30 Group Size = 60 Group Size = 30 Group Size = 60 Quality of Indicators Quality of Indicators Quality of Indicators Quality of Indicators    0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 12 (1, 1) 1 -0.63 -0.64 -0.29 -0.06 -0.65 -0.68 0.12 -0.71 0.03 -0.44 -0.81 0.05 1.33 -0.57 -0.56 0.03 -0.68 -0.64 -0.64 -0.53 -0.68 0.03 -0.65 -0.79 0.05 1.5 -0.21 -0.57 0.03 -0.66 -0.60 -0.71 -0.04 -0.65 0.03 -0.48 -0.80 0.04 1.75 -0.09 -0.69 0.02 -0.50 -0.64 -0.63 -0.51 -0.61 0.03 -0.63 -0.82 0.05 (1.33, 2) 1 -0.21 -0.51 -0.47 -0.68 -0.62 -0.53 -0.11 -0.69 0.04 276.67 -0.77 0.04 1.33 3.82 -0.55 -0.56 -0.61 -0.58 0.04 -0.58 -0.68 0.04 37.60 -0.77 0.04 1.5 -0.42 -0.51 -0.56 -0.65 -0.58 0.04 0.68 -0.66 0.04 -0.34 -0.78 0.05 1.75 -0.57 -0.45 -0.51 -0.62 -0.60 -0.38 2.47 -0.65 0.03 -0.36 -0.78 0.05 (1.5, 3) 1 -0.16 -0.54 -0.50 -0.65 -0.47 -0.37 -0.47 -0.65 0.05 9.76 -0.76 0.04 1.33 -0.29 -0.60 -0.50 -0.46 -0.56 -0.37 0.34 -0.57 0.04 -0.22 -0.73 0.04 1.5 -0.08 -0.55 -0.54 3.89 -0.56 0.03 -0.45 -0.65 0.04 3.76 -0.73 0.05 1.75 1.61 -0.57 -0.53 -0.45 -0.58 0.03 -0.27 -0.64 -0.45 0.01 -0.69 0.05 (1.75, 5) 1 -0.59 -0.54 -0.70 -0.65 -0.55 -0.62 -0.46 -0.61 -0.69 -0.07 -0.68 -0.80 1.33 -0.68 -0.50 -0.66 4.91 -0.54 -0.49 -0.51 -0.66 0.04 -0.29 -0.74 -0.81 1.5 -0.64 -0.47 -0.68 -0.68 -0.56 -0.57 -0.39 -0.68 -0.64 -0.42 -0.75 -0.81 1.75 -0.71 -0.52 -0.60 -0.56 -0.56 -0.68 -0.30 -0.66 -0.75 -0.48 -0.73 -0.81    156  Table 33. 95% Confidence Interval Coverage Rates of Level-1 Covariate Effect on the Latent Class 1 by Study Factors Number  of Indicators Level-1  Covariate Effect Level-2  Covariate Effect Number of Groups = 50 Number of Groups = 150 Group Size = 30 Group Size = 60 Group Size = 30 Group Size = 60 Quality of Indicators Quality of Indicators Quality of Indicators Quality of Indicators    0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 6 (1, 1) 1 0.76 0.87 0.94 0.77 0.97 0.98 0.86 0.98 0.97 0.91 0.93 0.89 1.33 0.68 0.88 0.95 0.80 0.95 0.96 0.87 0.98 0.96 0.88 0.94 0.88 1.5 0.76 0.89 0.96 0.73 0.93 0.96 0.89 0.97 0.97 0.90 0.95 0.89 1.75 0.74 0.93 0.97 0.79 0.94 0.96 0.91 0.99 0.97 0.90 0.93 0.90 (1.33, 2) 1 0.68 0.86 0.86 0.74 0.81 0.77 0.78 0.82 0.81 0.82 0.63 0.79 1.33 0.67 0.85 0.83 0.69 0.83 0.77 0.82 0.82 0.77 0.87 0.63 0.79 1.5 0.67 0.87 0.78 0.66 0.84 0.76 0.80 0.83 0.77 0.78 0.58 0.71 1.75 0.70 0.89 0.78 0.67 0.86 0.77 0.77 0.83 0.79 0.76 0.51 0.76 (1.5, 3) 1 0.71 0.75 0.75 0.68 0.66 0.75 0.70 0.70 0.73 0.80 0.52 0.71 1.33 0.71 0.76 0.77 0.64 0.76 0.70 0.67 0.76 0.66 0.82 0.51 0.71 1.5 0.72 0.78 0.69 0.64 0.75 0.69 0.66 0.74 0.70 0.81 0.55 0.70 1.75 0.70 0.83 0.76 0.65 0.74 0.70 0.74 0.72 0.69 0.83 0.56 0.73 (1.75, 5) 1 0.63 0.73 0.71 0.56 0.76 0.66 0.72 0.71 0.70 0.70 0.40 0.68 1.33 0.61 0.73 0.72 0.59 0.77 0.66 0.70 0.66 0.67 0.71 0.48 0.70 1.5 0.57 0.73 0.74 0.60 0.77 0.63 0.71 0.73 0.64 0.79 0.50 0.66 1.75 0.54 0.75 0.71 0.60 0.77 0.61 0.78 0.67 0.70 0.78 0.47 0.62    157  Table 33. (cont™d) Number  of Indicators Level-1  Covariate Effect Level-2  Covariate Effect Number of Groups = 50 Number of Groups = 150 Group Size = 30 Group Size = 60 Group Size = 30 Group Size = 60 Quality of Indicators Quality of Indicators Quality of Indicators Quality of Indicators    0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 12 (1, 1) 1 0.82 0.94 0.89 0.91 0.97 0.98 0.90 0.93 0.95 0.94 0.97 0.99 1.33 0.84 0.96 0.90 0.92 0.99 0.98 0.95 0.95 0.96 0.96 0.97 0.99 1.5 0.84 0.96 0.89 0.93 0.99 0.98 0.94 0.95 0.96 0.94 0.97 0.99 1.75 0.80 0.93 0.89 0.92 0.99 0.98 0.95 0.94 0.96 0.96 0.96 0.99 (1.33, 2) 1 0.79 0.93 0.89 0.93 0.80 0.93 0.89 0.74 0.92 0.89 0.77 0.97 1.33 0.82 0.94 0.87 0.89 0.79 0.96 0.91 0.72 0.93 0.87 0.77 0.96 1.5 0.80 0.94 0.88 0.89 0.78 0.95 0.91 0.76 0.93 0.87 0.77 0.96 1.75 0.82 0.95 0.89 0.87 0.83 0.95 0.92 0.78 0.95 0.89 0.74 0.96 (1.5, 3) 1 0.81 0.85 0.88 0.86 0.84 0.93 0.89 0.75 0.94 0.83 0.79 0.96 1.33 0.79 0.81 0.91 0.85 0.76 0.94 0.87 0.82 0.95 0.81 0.82 0.96 1.5 0.76 0.82 0.91 0.88 0.78 0.94 0.85 0.76 0.93 0.86 0.81 0.96 1.75 0.79 0.84 0.92 0.87 0.77 0.94 0.86 0.76 0.93 0.82 0.85 0.96 (1.75, 5) 1 0.78 0.81 0.83 0.75 0.81 0.88 0.80 0.83 0.90 0.85 0.88 0.87 1.33 0.76 0.86 0.84 0.77 0.79 0.90 0.83 0.77 0.95 0.80 0.83 0.86 1.5 0.75 0.84 0.83 0.76 0.80 0.89 0.80 0.76 0.91 0.80 0.80 0.87 1.75 0.75 0.83 0.88 0.77 0.79 0.86 0.85 0.78 0.88 0.76 0.83 0.86    158  Table 34. 95% Confidence Interval Coverage Rates of Level-1 Covariate Effect on the Latent Class 2 by Study Factors Number  of Indicators Level-1  Covariate Effect Level-2  Covariate Effect Number of Groups = 50 Number of Groups = 150 Group Size = 30 Group Size = 60 Group Size = 30 Group Size = 60 Quality of Indicators Quality of Indicators Quality of Indicators Quality of Indicators    0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 6 (1, 1) 1 0.68 0.90 0.96 0.62 0.93 0.98 0.77 0.96 0.98 0.84 0.96 0.97 1.33 0.64 0.92 0.95 0.68 0.92 0.98 0.74 0.98 0.98 0.88 0.97 0.97 1.5 0.67 0.90 0.96 0.68 0.94 0.96 0.76 0.97 0.97 0.86 0.97 0.97 1.75 0.65 0.92 0.95 0.66 0.95 0.96 0.82 0.96 0.97 0.81 0.97 0.98 (1.33, 2) 1 0.58 0.80 0.80 0.63 0.78 0.72 0.68 0.78 0.80 0.76 0.62 0.80 1.33 0.59 0.79 0.73 0.58 0.80 0.76 0.66 0.78 0.79 0.78 0.61 0.79 1.5 0.58 0.80 0.72 0.57 0.74 0.74 0.67 0.75 0.78 0.73 0.61 0.72 1.75 0.62 0.73 0.74 0.60 0.70 0.77 0.62 0.76 0.80 0.74 0.54 0.76 (1.5, 3) 1 0.56 0.70 0.70 0.60 0.68 0.74 0.55 0.61 0.70 0.71 0.53 0.69 1.33 0.57 0.70 0.73 0.57 0.70 0.65 0.54 0.67 0.64 0.78 0.54 0.68 1.5 0.59 0.66 0.66 0.56 0.67 0.64 0.54 0.68 0.68 0.72 0.58 0.68 1.75 0.58 0.71 0.66 0.58 0.65 0.67 0.64 0.72 0.69 0.75 0.59 0.72 (1.75, 5) 1 0.43 0.65 0.66 0.46 0.73 0.69 0.47 0.65 0.69 0.65 0.40 0.64 1.33 0.44 0.67 0.67 0.46 0.73 0.69 0.44 0.65 0.65 0.65 0.44 0.64 1.5 0.45 0.63 0.68 0.44 0.68 0.64 0.47 0.72 0.65 0.71 0.50 0.63 1.75 0.43 0.65 0.68 0.44 0.71 0.62 0.51 0.69 0.72 0.68 0.45 0.58    159  Table 34. (cont™d) Number  of Indicators Level-1  Covariate Effect Level-2  Covariate Effect Number of Groups = 50 Number of Groups = 150 Group Size = 30 Group Size = 60 Group Size = 30 Group Size = 60 Quality of Indicators Quality of Indicators Quality of Indicators Quality of Indicators    0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 12 (1, 1) 1 0.82 0.92 0.93 0.91 0.96 0.95 0.94 0.95 0.93 0.91 0.97 0.98 1.33 0.84 0.94 0.94 0.95 0.98 0.96 0.96 0.95 0.89 0.94 0.99 0.98 1.5 0.85 0.93 0.94 0.93 0.98 0.96 0.92 0.94 0.90 0.94 0.99 0.98 1.75 0.81 0.97 0.94 0.87 0.96 0.96 0.95 0.95 0.91 0.94 0.99 0.97 (1.33, 2) 1 0.71 0.86 0.94 0.80 0.88 0.92 0.75 0.79 0.88 0.65 0.79 0.92 1.33 0.72 0.83 0.94 0.82 0.89 0.95 0.75 0.88 0.89 0.72 0.83 0.92 1.5 0.67 0.82 0.93 0.77 0.86 0.95 0.81 0.81 0.89 0.70 0.82 0.92 1.75 0.71 0.82 0.92 0.77 0.88 0.94 0.81 0.85 0.92 0.73 0.79 0.92 (1.5, 3) 1 0.61 0.84 0.90 0.70 0.87 0.93 0.75 0.83 0.90 0.67 0.80 0.86 1.33 0.64 0.82 0.91 0.65 0.86 0.91 0.69 0.86 0.90 0.73 0.81 0.85 1.5 0.60 0.77 0.90 0.68 0.86 0.92 0.69 0.80 0.89 0.77 0.82 0.84 1.75 0.60 0.82 0.90 0.70 0.85 0.91 0.67 0.81 0.87 0.76 0.83 0.86 (1.75, 5) 1 0.52 0.85 0.77 0.60 0.86 0.85 0.70 0.85 0.87 0.69 0.79 0.80 1.33 0.51 0.87 0.82 0.57 0.87 0.87 0.72 0.80 0.93 0.71 0.76 0.79 1.5 0.50 0.87 0.78 0.57 0.85 0.86 0.69 0.78 0.91 0.75 0.74 0.79 1.75 0.50 0.82 0.84 0.56 0.88 0.82 0.69 0.82 0.87 0.69 0.77 0.81    160  Table 35. 95% Confidence Interval Coverage Rates of Level-2 Covariate Effect by Study Factors Number  of Indicators Level-1  Covariate Effect Level-2  Covariate Effect Number of Groups = 50 Number of Groups = 150 Group Size = 30 Group Size = 60 Group Size = 30 Group Size = 60 Quality of Indicators Quality of Indicators Quality of Indicators Quality of Indicators    0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 6 (1, 1) 1 0.41 0.83 0.97 0.55 0.93 0.96 0.66 0.96 0.95 0.88 0.96 0.97 1.33 0.39 0.79 0.97 0.51 0.93 0.96 0.71 0.92 0.92 0.89 0.93 0.92 1.5 0.39 0.76 0.97 0.52 0.93 0.94 0.68 0.85 0.88 0.85 0.92 0.85 1.75 0.43 0.78 0.93 0.53 0.93 0.91 0.68 0.86 0.75 0.87 0.80 0.80 (1.33, 2) 1 0.40 0.83 0.98 0.55 0.96 0.97 0.61 0.96 0.94 0.87 0.98 0.97 1.33 0.44 0.79 0.98 0.53 0.97 0.97 0.65 0.93 0.88 0.86 0.95 0.93 1.5 0.40 0.77 0.96 0.51 0.96 0.95 0.61 0.89 0.83 0.88 0.88 0.90 1.75 0.40 0.78 0.95 0.48 0.92 0.94 0.63 0.90 0.73 0.84 0.78 0.73 (1.5, 3) 1 0.43 0.71 0.98 0.54 0.96 0.98 0.60 0.96 0.95 0.83 0.98 0.95 1.33 0.40 0.69 0.94 0.58 0.96 0.96 0.58 0.93 0.92 0.80 0.91 0.91 1.5 0.39 0.69 0.92 0.52 0.95 0.95 0.66 0.91 0.88 0.80 0.91 0.85 1.75 0.36 0.71 0.90 0.54 0.95 0.90 0.64 0.87 0.77 0.74 0.79 0.73 (1.75, 5) 1 0.41 0.73 0.97 0.42 0.99 0.98 0.60 0.98 0.97 0.80 0.95 0.98 1.33 0.44 0.76 0.96 0.48 0.98 0.94 0.60 0.92 0.96 0.83 0.90 0.95 1.5 0.45 0.73 0.91 0.48 0.94 0.90 0.59 0.89 0.92 0.81 0.88 0.90 1.75 0.45 0.74 0.91 0.51 0.94 0.90 0.56 0.87 0.78 0.81 0.82 0.73    161  Table 35. (cont™d) Number  of Indicators Level-1  Covariate Effect Level-2  Covariate Effect Number of Groups = 50 Number of Groups = 150 Group Size = 30 Group Size = 60 Group Size = 30 Group Size = 60 Quality of Indicators Quality of Indicators Quality of Indicators Quality of Indicators    0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 12 (1, 1) 1 0.49 0.94 0.95 0.77 0.94 0.94 0.87 0.95 0.96 0.97 0.97 0.97 1.33 0.52 0.91 0.91 0.79 0.94 0.93 0.87 0.92 0.88 0.94 0.92 0.92 1.5 0.51 0.92 0.87 0.72 0.95 0.90 0.82 0.85 0.84 0.92 0.88 0.86 1.75 0.52 0.89 0.84 0.81 0.89 0.88 0.80 0.78 0.70 0.81 0.73 0.70 (1.33, 2) 1 0.55 0.92 0.96 0.81 0.95 0.94 0.84 0.95 0.96 0.99 0.97 0.99 1.33 0.56 0.90 0.92 0.79 0.94 0.92 0.84 0.94 0.90 0.95 0.94 0.91 1.5 0.51 0.89 0.88 0.79 0.92 0.91 0.84 0.90 0.89 0.95 0.86 0.84 1.75 0.54 0.87 0.85 0.76 0.87 0.89 0.80 0.80 0.73 0.91 0.73 0.65 (1.5, 3) 1 0.54 0.96 0.94 0.77 0.98 0.94 0.82 0.96 0.95 0.99 0.98 0.99 1.33 0.47 0.94 0.91 0.71 0.94 0.92 0.79 0.96 0.91 0.97 0.93 0.93 1.5 0.47 0.89 0.89 0.74 0.93 0.91 0.85 0.92 0.86 0.94 0.85 0.87 1.75 0.46 0.89 0.86 0.76 0.88 0.87 0.80 0.83 0.76 0.88 0.77 0.71 (1.75, 5) 1 0.47 0.96 0.92 0.78 0.96 0.93 0.82 0.95 0.95 0.98 0.98 0.98 1.33 0.44 0.90 0.90 0.76 0.96 0.94 0.79 0.95 0.91 0.93 0.93 0.94 1.5 0.40 0.92 0.87 0.73 0.91 0.93 0.75 0.92 0.90 0.91 0.87 0.90 1.75 0.44 0.92 0.83 0.81 0.91 0.89 0.72 0.78 0.78 0.87 0.67 0.75    162  Table 36. 95% Confidence Interval Coverage Rates of Conditional Response Probability of Latent Class 1 by Study Factors Number  of Indicators Level-1  Covariate Effect Level-2  Covariate Effect Number of Groups = 50 Number of Groups = 150 Group Size = 30 Group Size = 60 Group Size = 30 Group Size = 60 Quality of Indicators Quality of Indicators Quality of Indicators Quality of Indicators    0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 6 (1, 1) 1 0.64 0.76 0.83 0.64 0.72 0.75 0.69 0.77 0.80 0.69 0.57 0.77 1.33 0.62 0.76 0.83 0.66 0.78 0.72 0.61 0.67 0.80 0.72 0.59 0.85 1.5 0.63 0.77 0.76 0.66 0.76 0.73 0.63 0.69 0.78 0.72 0.65 0.81 1.75 0.61 0.78 0.80 0.64 0.80 0.78 0.66 0.76 0.79 0.73 0.67 0.79 (1.33, 2) 1 0.66 0.72 0.78 0.64 0.75 0.73 0.73 0.78 0.81 0.75 0.58 0.82 1.33 0.66 0.76 0.71 0.63 0.77 0.76 0.71 0.77 0.80 0.78 0.58 0.80 1.5 0.65 0.78 0.74 0.62 0.77 0.75 0.70 0.78 0.79 0.76 0.55 0.73 1.75 0.63 0.79 0.76 0.64 0.76 0.77 0.70 0.76 0.80 0.71 0.50 0.77 (1.5, 3) 1 0.68 0.75 0.73 0.70 0.72 0.76 0.77 0.68 0.74 0.80 0.54 0.76 1.33 0.66 0.75 0.77 0.69 0.72 0.70 0.75 0.70 0.68 0.80 0.48 0.74 1.5 0.66 0.74 0.69 0.68 0.69 0.69 0.77 0.67 0.74 0.79 0.50 0.76 1.75 0.66 0.78 0.71 0.66 0.69 0.71 0.77 0.71 0.73 0.81 0.56 0.77 (1.75, 5) 1 0.68 0.70 0.71 0.67 0.72 0.71 0.75 0.63 0.75 0.82 0.38 0.76 1.33 0.70 0.76 0.74 0.66 0.74 0.69 0.73 0.60 0.72 0.77 0.42 0.74 1.5 0.67 0.75 0.73 0.66 0.67 0.65 0.75 0.69 0.70 0.84 0.46 0.72 1.75 0.67 0.77 0.71 0.67 0.74 0.65 0.74 0.61 0.75 0.84 0.43 0.69    163  Table 36. (cont™d) Number  of Indicators Level-1  Covariate Effect Level-2  Covariate Effect Number of Groups = 50 Number of Groups = 150 Group Size = 30 Group Size = 60 Group Size = 30 Group Size = 60 Quality of Indicators Quality of Indicators Quality of Indicators Quality of Indicators    0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 12 (1, 1) 1 0.68 0.68 0.94 0.75 0.68 0.91 0.72 0.67 0.96 0.70 0.67 0.95 1.33 0.69 0.71 0.95 0.74 0.69 0.92 0.72 0.73 0.96 0.64 0.72 0.95 1.5 0.74 0.67 0.94 0.77 0.77 0.90 0.73 0.73 0.96 0.69 0.70 0.95 1.75 0.71 0.65 0.95 0.75 0.67 0.92 0.68 0.76 0.96 0.64 0.68 0.95 (1.33, 2) 1 0.72 0.77 0.93 0.73 0.72 0.92 0.73 0.71 0.95 0.73 0.77 0.95 1.33 0.77 0.79 0.91 0.75 0.74 0.95 0.74 0.69 0.95 0.70 0.76 0.95 1.5 0.75 0.75 0.91 0.78 0.74 0.95 0.75 0.73 0.95 0.77 0.75 0.95 1.75 0.75 0.78 0.92 0.75 0.74 0.94 0.75 0.73 0.96 0.72 0.75 0.95 (1.5, 3) 1 0.72 0.72 0.92 0.78 0.83 0.94 0.80 0.76 0.96 0.78 0.79 0.95 1.33 0.73 0.75 0.93 0.74 0.76 0.94 0.78 0.83 0.96 0.76 0.82 0.95 1.5 0.73 0.72 0.93 0.74 0.77 0.94 0.79 0.77 0.96 0.80 0.82 0.95 1.75 0.72 0.76 0.93 0.75 0.74 0.94 0.77 0.78 0.95 0.76 0.86 0.95 (1.75, 5) 1 0.73 0.70 0.84 0.75 0.79 0.91 0.81 0.82 0.93 0.73 0.86 0.93 1.33 0.71 0.78 0.88 0.77 0.78 0.92 0.79 0.76 0.95 0.70 0.82 0.91 1.5 0.71 0.79 0.87 0.76 0.78 0.91 0.81 0.74 0.94 0.76 0.79 0.92 1.75 0.71 0.78 0.90 0.76 0.78 0.90 0.77 0.76 0.92 0.73 0.81 0.92    164  Table 37. 95% Confidence Interval Coverage Rates of Conditional Response Probability of Latent Class 2 by Study Factors Number  of Indicators Level-1  Covariate Effect Level-2  Covariate Effect Number of Groups = 50 Number of Groups = 150 Group Size = 30 Group Size = 60 Group Size = 30 Group Size = 60 Quality of Indicators Quality of Indicators Quality of Indicators Quality of Indicators    0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 6 (1, 1) 1 0.73 0.70 0.83 0.68 0.77 0.84 0.69 0.80 0.88 0.79 0.79 0.85 1.33 0.70 0.69 0.84 0.72 0.78 0.82 0.70 0.76 0.87 0.78 0.82 0.89 1.5 0.71 0.69 0.82 0.74 0.79 0.81 0.70 0.75 0.86 0.78 0.82 0.88 1.75 0.71 0.71 0.82 0.72 0.78 0.85 0.70 0.77 0.87 0.79 0.81 0.86 (1.33, 2) 1 0.73 0.76 0.82 0.73 0.79 0.84 0.73 0.82 0.89 0.82 0.78 0.88 1.33 0.70 0.76 0.79 0.72 0.79 0.85 0.70 0.80 0.87 0.80 0.80 0.87 1.5 0.69 0.76 0.80 0.74 0.76 0.84 0.72 0.83 0.87 0.80 0.76 0.83 1.75 0.69 0.74 0.81 0.74 0.75 0.86 0.70 0.82 0.88 0.78 0.73 0.86 (1.5, 3) 1 0.73 0.77 0.82 0.79 0.76 0.86 0.75 0.78 0.84 0.83 0.77 0.85 1.33 0.72 0.74 0.84 0.76 0.79 0.82 0.74 0.77 0.80 0.82 0.78 0.82 1.5 0.73 0.74 0.79 0.74 0.77 0.82 0.70 0.79 0.83 0.84 0.78 0.83 1.75 0.73 0.75 0.81 0.75 0.78 0.83 0.76 0.81 0.82 0.83 0.76 0.83 (1.75, 5) 1 0.73 0.81 0.81 0.78 0.79 0.81 0.81 0.82 0.83 0.88 0.73 0.82 1.33 0.73 0.80 0.82 0.75 0.81 0.81 0.80 0.80 0.81 0.85 0.78 0.82 1.5 0.73 0.80 0.81 0.76 0.79 0.78 0.81 0.84 0.81 0.86 0.77 0.80 1.75 0.75 0.80 0.80 0.77 0.80 0.77 0.80 0.82 0.84 0.85 0.75 0.79    165  Table 37. (cont™d) Number  of Indicators Level-1  Covariate Effect Level-2  Covariate Effect Number of Groups = 50 Number of Groups = 150 Group Size = 30 Group Size = 60 Group Size = 30 Group Size = 60 Quality of Indicators Quality of Indicators Quality of Indicators Quality of Indicators    0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 12 (1, 1) 1 0.78 0.84 0.94 0.80 0.95 0.96 0.79 0.94 0.96 0.81 0.93 0.95 1.33 0.81 0.84 0.94 0.77 0.93 0.96 0.83 0.94 0.96 0.81 0.92 0.95 1.5 0.80 0.81 0.94 0.76 0.93 0.96 0.79 0.93 0.96 0.82 0.93 0.95 1.75 0.78 0.83 0.94 0.80 0.93 0.96 0.83 0.92 0.96 0.81 0.92 0.95 (1.33, 2) 1 0.82 0.87 0.94 0.81 0.96 0.94 0.83 0.92 0.95 0.82 0.96 0.95 1.33 0.81 0.85 0.93 0.82 0.94 0.96 0.85 0.93 0.95 0.83 0.97 0.96 1.5 0.81 0.84 0.93 0.79 0.95 0.95 0.80 0.93 0.95 0.82 0.96 0.95 1.75 0.81 0.85 0.93 0.80 0.95 0.96 0.82 0.92 0.96 0.81 0.95 0.96 (1.5, 3) 1 0.83 0.89 0.93 0.80 0.96 0.95 0.85 0.94 0.96 0.81 0.97 0.96 1.33 0.83 0.89 0.93 0.79 0.94 0.94 0.80 0.94 0.96 0.82 0.96 0.96 1.5 0.83 0.88 0.93 0.81 0.95 0.95 0.81 0.95 0.96 0.83 0.97 0.96 1.75 0.83 0.89 0.93 0.81 0.96 0.94 0.82 0.94 0.95 0.85 0.96 0.95 (1.75, 5) 1 0.83 0.91 0.89 0.85 0.97 0.92 0.86 0.96 0.93 0.84 0.97 0.93 1.33 0.83 0.90 0.91 0.84 0.96 0.93 0.86 0.95 0.95 0.84 0.96 0.94 1.5 0.83 0.92 0.90 0.84 0.96 0.92 0.86 0.95 0.94 0.86 0.96 0.93 1.75 0.82 0.92 0.91 0.85 0.96 0.91 0.84 0.95 0.92 0.84 0.97 0.93    166  Table 38. 95% Confidence Interval Coverage Rates of Conditional Response Probability of Latent Class 3 by Study Factors Number  of Indicators Level-1  Covariate Effect Level-2  Covariate Effect Number of Groups = 50 Number of Groups = 150 Group Size = 30 Group Size = 60 Group Size = 30 Group Size = 60 Quality of Indicators Quality of Indicators Quality of Indicators Quality of Indicators    0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 6 (1, 1) 1 0.65 0.66 0.83 0.72 0.70 0.78 0.68 0.72 0.86 0.67 0.63 0.83 1.33 0.61 0.68 0.84 0.69 0.79 0.79 0.62 0.67 0.88 0.69 0.63 0.87 1.5 0.66 0.69 0.81 0.70 0.77 0.77 0.63 0.65 0.86 0.67 0.66 0.84 1.75 0.64 0.70 0.80 0.72 0.80 0.83 0.64 0.71 0.86 0.66 0.71 0.85 (1.33, 2) 1 0.64 0.67 0.83 0.71 0.70 0.83 0.66 0.76 0.88 0.72 0.71 0.88 1.33 0.63 0.69 0.77 0.65 0.74 0.84 0.63 0.75 0.87 0.69 0.68 0.87 1.5 0.66 0.70 0.79 0.68 0.70 0.83 0.62 0.75 0.87 0.74 0.69 0.85 1.75 0.64 0.67 0.81 0.68 0.67 0.84 0.64 0.75 0.87 0.71 0.65 0.87 (1.5, 3) 1 0.67 0.71 0.80 0.71 0.70 0.87 0.67 0.68 0.84 0.72 0.63 0.85 1.33 0.66 0.66 0.80 0.71 0.68 0.83 0.60 0.70 0.83 0.77 0.64 0.83 1.5 0.66 0.69 0.75 0.71 0.65 0.82 0.67 0.71 0.87 0.75 0.64 0.83 1.75 0.66 0.70 0.74 0.72 0.66 0.83 0.67 0.72 0.86 0.73 0.72 0.86 (1.75, 5) 1 0.67 0.68 0.82 0.65 0.73 0.84 0.71 0.67 0.86 0.70 0.59 0.87 1.33 0.68 0.68 0.84 0.66 0.73 0.83 0.71 0.65 0.83 0.68 0.55 0.86 1.5 0.66 0.71 0.80 0.66 0.68 0.81 0.74 0.75 0.84 0.70 0.60 0.86 1.75 0.65 0.71 0.80 0.69 0.74 0.82 0.71 0.68 0.86 0.68 0.59 0.85    167  Table 38. (cont™d) Number of Indicators Level-1 Covariate Effect Level-2 Covariate Effect Number of Groups = 50 Number of Groups = 150 Group Size = 30 Group Size = 60 Group Size = 30 Group Size = 60 Quality of Indicators Quality of Indicators Quality of Indicators Quality of Indicators    0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 0.6 0.7 0.8 12 (1, 1) 1 0.65 0.69 0.94 0.68 0.69 0.91 0.68 0.69 0.96 0.64 0.69 0.96 1.33 0.65 0.69 0.95 0.70 0.66 0.92 0.69 0.74 0.96 0.62 0.74 0.96 1.5 0.66 0.69 0.95 0.70 0.74 0.90 0.69 0.74 0.96 0.67 0.71 0.96 1.75 0.68 0.63 0.95 0.64 0.68 0.92 0.66 0.79 0.96 0.64 0.68 0.96 (1.33, 2) 1 0.68 0.75 0.93 0.70 0.70 0.94 0.72 0.73 0.96 0.69 0.77 0.96 1.33 0.67 0.77 0.91 0.72 0.75 0.96 0.67 0.72 0.95 0.68 0.75 0.97 1.5 0.69 0.74 0.91 0.72 0.75 0.96 0.73 0.77 0.95 0.71 0.74 0.97 1.75 0.68 0.75 0.92 0.70 0.72 0.95 0.67 0.77 0.96 0.70 0.74 0.96 (1.5, 3) 1 0.68 0.71 0.92 0.72 0.84 0.95 0.70 0.79 0.96 0.76 0.78 0.96 1.33 0.72 0.73 0.91 0.67 0.76 0.95 0.68 0.86 0.96 0.77 0.82 0.97 1.5 0.68 0.67 0.90 0.70 0.76 0.96 0.72 0.78 0.96 0.79 0.81 0.97 1.75 0.67 0.71 0.90 0.71 0.74 0.96 0.68 0.80 0.95 0.73 0.85 0.96 (1.75, 5) 1 0.74 0.68 0.81 0.71 0.80 0.91 0.73 0.82 0.92 0.75 0.86 0.91 1.33 0.71 0.77 0.85 0.70 0.79 0.93 0.76 0.77 0.96 0.71 0.81 0.90 1.5 0.70 0.77 0.82 0.73 0.78 0.92 0.74 0.74 0.93 0.74 0.78 0.91 1.75 0.69 0.75 0.88 0.73 0.77 0.90 0.77 0.76 0.89 0.75 0.81 0.91 168      
    REFERENCES   169  REFERENCES   Akaike, H. (1974). A new look at the statistical model identification. Automatic Control, IEEE Transactions on. Retrieved from http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=1100705 Asparouhov, T., & Muthén, B. (2008). Multilevel mixture models. In G. R. Hancock & K. M. Samuelsen (Eds.), Advances in Latent Variable Mixture Models (pp. 27Œ51). Charlotte, NC: Information Age Publishing. Bakk, Z., Tekle, F. B., & Vermunt, J. K. (2013). Estimating the association between latent class membership and external variables using bias-adjusted three-step approaches. Sociological Methodology, 43(1), 272Œ311. http://doi.org/10.1177/0081175012470644 Bandeen-Roche, K., Miglioretti, D. L., Zeger, S. L., & Rathouz, P. J. (1997). Latent variable regression for multiple discrete outcomes. Journal of the American Statistical Association, 
92(440), 1375Œ1386. http://doi.org/10.2307/2965407 Bolck, A., Croon, M., & Hagenaars, J. (2004). Estimating latent structure models with categorical variables: 1-step versus 3-step estimators. Political Analysis, 12(1), 3Œ27. http://doi.org/10.1093/pan/mph001 Bozdogan, H. (1987). Model selection and Akaike™s Information Criterion (AIC): The general theory and its analytical extensions. Psychometrika, 52(3), 345Œ370. http://doi.org/10.1007/BF02294361 Bozdogan, H. (1993). Choosing the number of component clusters in the mixture-model using a new informational complexity criterion of the inverse-Fisher information matrix. In O. Opitz, B. Lausen, & R. Klar (Eds.), Information and Classificatoin, Concepts, Methods and 
Applications (pp. 40Œ54). Berlin, Germany: Springer. Bray, B. C., Lanza, S. T., & Tan, X. (2015). Eliminating bias in classify-analyze approaches for latent class analysis. Structural Equation Modeling: A Multidisciplinary Journal, 22(1), 1Œ11. http://doi.org/10.1080/10705511.2014.935265 Clogg, C. C. (1995). Latent class models. In G. Arminger, C. C. Clogg, & M. E. Sobel (Eds.), Handbook of Statistical Modeling for the Social and Behavioral Sciences (pp. 311Œ359). New York, NY: Plenum Press. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: 170  Erlbaum. Collins, L., & Lanza, S. (2010). Latent class and latent transition analysis: With applications in the social, behavioral, and health sciences. Retrieved from http://books.google.com/books?hl=en&lr=&id=3xJaoIqhZhEC&oi=fnd&pg=PR7&dq=latent+transition+model&ots=9Vb2GuK21Y&sig=qdq9ViUi6FiL_nFn6HJ-dy3gjWA Everitt, B. S., & Hand, D. J. (1981). Finite mixture distributions. New York, N.Y.: Chapman and Hall. Finch, W. H., & French, B. F. (2013). Multilevel latent class analysis: Parametric and nonparametric models. The Journal of Experimental Education, 82(3), 307Œ333. 
http://doi.org/10.1080/00220973.2013.813361 Henry, K. L., & Muthén, B. (2010). Multilevel latent class analysis: An application of adolescent smoking typologies with individual and contextual predictors. Structural Equation 
Modeling: A Multidisciplinary Journal, 17(2), 193Œ215. http://doi.org/10.1080/10705511003659342 Kaplan, D., & Keller, B. (2011). A note on cluster effects in latent class analysis. Structural Equation Modeling: A Multidisciplinary Journal, 18(4), 525Œ536. http://doi.org/10.1080/10705511.2011.607071 Lazarsfeld, P. F., & Henry, N. W. (1968). Latent structure analysis. Boston, MA: Houghton Mifflin. number of lower- and higher-level classes in multilevel latent class analysis. Sociological Methodology, 40(1), 247Œ283. http://doi.org/10.1111/j.1467-9531.2010.01231.x Marsh, H. W., Hau, K.-T., Balla, J. R., & Grayson, D. (1998). Is more ever too much: The number of indicators per factor in confirmatory factor analysis. Multivariate Behavioral 
Research, 33(2), 181Œ220. http://doi.org/10.1207/s15327906mbr3302_1 McCutcheon, A. L. (1987). Latent Class Analysis. Newbury Park, CA: Sage Publications. Mclachlan, G., & Peel, D. (2000). Finite mixture models. John Wiley & Sons, Inc. Muthén, B., & Asparouhov, T. (2009a). Multilevel regression mixture analysis. Journal of the Royal Statistical Society: Series A (Statistics in Society), 172(3), 639Œ657. http://doi.org/10.1111/j.1467-985X.2009.00589.x Muthén, B., & Asparouhov, T. (2009b). Multilevel regression mixture analysis. Journal of the 171  Royal Statistical Society . Series A ( Statistics in Society ), 172(3, Recent Advances in Multilevel Modelling Methodology and Aplications), 639Œ657. Muthén, L. K., & Muthén, B. O. (n.d.). Mplus User™s Guide (7th Ed.). Los Angeles, CA: Muthén & Muthén. Muthén, L. K., & Muthén, B. O. (2002). How to use a Monte Carlo study to decide on sample size and determine power. Structural Equation Modeling: A Multidisciplinary Journal, 9(4), 599Œ620. http://doi.org/10.1207/S15328007SEM0904 Nagin, D. S. (2005). Linking group membership to covariates. In Group-Based Modeling of Development. Cambridge: Harvard University Press. Reboussin, B. A., & Anthony, J. C. (2001). Latent class marginal regression models for modelling youthful drug involvement and its suspected influences. Statistics in Medicine, 20(4), 623Œ639. http://doi.org/10.1002/sim.695 Robinson, W. S. (1950). Ecological correlations and the behavior individuals. American Sociological Review, 15(3), 351Œ357. Rubin, D. B. (1987). Multiple Imputation for Nonresponse in Surveys. New York: J. Wiley & Sons. Schlattmann, P. (2009). Medical applications of finite mixture models. Springer-Verlag Berlin Heidelberg. Schuler, M. S., Leoutsakos, J. M., & Stuart, E. A. (2014). Addressing confounding when estimating the effects of latent classes on a distal outcome. Health Services and Outcomes Research Methodology, 232Œ254. http://doi.org/10.1007/s10742-014-0122-0 Schwarz, G. (1978). Estimating the dimension of a model. The Annals of Statistics, 6(2), 461Œ464. Team, R. C. (2016). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. Retrieved from https://www.r-project.org/ Titterington, D. M., Smith, A. F. M., & Makov, U. E. (1985). Statistical analysis of finite mixture distributions. John Wiley & Sons Inc. Vermunt, J. K. (2003). Multilevel latent class models. Sociological Methodology, 33(Lc), 213Œ239. http://doi.org/10.1111/j.0081-1750.2003.t01-1-00131.x Vermunt, J. K. (2008). Latent class and finite mixture models for multilevel data sets. Statistical 172  Methods in Medical Research, 17(1), 33Œ51. http://doi.org/10.1177/0962280207081238 Vermunt, J. K. (2010). Latent class modeling with covariates: Two improved three-step approaches. Political Analysis, 18(4), 450Œ469. http://doi.org/10.1093/pan/mpq025 Vermunt, J. K., & Magidson, J. (2002). Latent class cluster analysis. In J. A. Hagenaars & A. L. McCutcheon (Eds.), Applied Latent Class Analysis (pp. 89Œ106). Cambridge: Cambridge University Press. Wang, J., & Wang, X. (2012). Mixture modeling. In Structural Equation Modeling: Applications Using Mplus (pp. 289Œ390). John Wiley & Sons, Inc. Wurpts, I. C., & Geiser, C. (2014). Is adding more indicators to a latent class analysis beneficial or detrimental? Results of a Monte-Carlo study. Frontiers in Psychology, 5(August), 1Œ15. http://doi.org/10.3389/fpsyg.2014.00920