Research investigators rely on information of intraclass correlation coefficients for planning and conducting designs and experiments for scientific inquiries in educational and social studies. Randomized controlled trials and cluster randomized studies are deemed as the gold standard for evidence-based interventions, and both approaches have been applied successfully in many situations for more effective decision-making in education and social research. The cluster randomized designs for community-based research, in particular, have been widely used in the modern era, since they are often operated at the group level, like a whole community or worksite, in order for researchers more easily to deal with random assignment of an entire intact group rather than that of each individual subject. Hence, such cluster-randomized trials or group-randomized experiments have become important and useful to provide evidence-guided practice models for scientific inquiry and research.The aim of this dissertation is to develop the methods for the intraclass correlation coefficients for binary and continuous outcomes in cluster-based intervention designs using hierarchal mixed model based on the scenarios of unconditional and conditional multilevel structures with cluster sampling schemes. Simulation studies are used to assess the statistical properties of intraclass correlation estimation and inference via the real data set of RSA-911 for people with disabilities served in the Michigan Rehabilitation Services Programs.The results show that the average (unadjusted) intraclass correlation is about 0.01 for competitive employment and about 0.02 for weekly earnings (quality employment) in Michigan. These average (unadjusted) intraclass correlations from RSA-911 are relatively low in comparison to education interventions or academic programs for assessments in reading and mathematics across K-12 (Bloom et al., 1999, 2007; Hedges & Hedberg, 2007; Schochet, 2008); however, they seem comparable to some extent from those psychological and mental health data in school-based intervention designs (Murray & Short, 1995).For future study, researchers may look into different types of integrated large-scale complex data sets such as RSA-911 data with a set of covariates from Census data for investigating how intraclass correlation performs in statistical estimation and inference across multiple platforms. In addition, it would be interesting to study how to deal with missing values in the estimation procedure of intraclass correlation, and what remedial procedure can be added to improve estimation process. For the proposed method, it would recommend the total sample size should be greater than 1,500 and within group sample size would be better to be larger than 100 (with the number of groups about 15).In conclusion, this study provides a comprehensive methodology for intraclass correlation estimation and inference using the mixed "analysis of variance" approach along with the derived sampling distribution (i.e., F-distribution) for testing hypothesis as well as building confidence interval on intraclass correlation estimates. Such proposed statistical procedures can be easily used and applied in any large-scale or small-scale data sets, whereas small total sample size and small within group size and missing data are limitations on intraclass correlation estimation in terms of precision and accuracy.
Read
