Based on Center for Disease Control and Prevention (CDC) surveillance reports, the prevalence of autism spectrum disorder (ASD) continues to increase (Maenner et al., 2021). As such, assessment tools that are efficient, cost-effective, and psychometrically sound are key to effective screening, accurate diagnosis, and clarification of intervention needs (Kuriakose & Shalev, 2016; Zwaigenbaum & Penner, 2018). The Gilliam Autism Rating Scale - Third Edition (GARS-3; Gilliam, 2013), a substantial revision from earlier editions, is a rating scale used to gather information from parents, caregivers, or teachers for screening or as part of a more comprehensive ASD assessment. Across editions, the GARS is considered a popular assessment tool among school psychologists (e.g., Aiello et al., 2017; Benson et al., 2019). However, despite the strong psychometric characteristics reported in the test manuals with standardization samples, prior editions were criticized for their screening performance in independent research samples, and factor analyses suggested problems with the test author's proposed subscales (e.g., Lecavalier, 2005; Pandolfi et al., 2010; South et al., 2002; Volker et al., 2016; Volker et al., 2022). To date, there has been little to no research focused on the psychometric properties of the current version of the GARS beyond what is reported in the test manual. Of critical importance, there have been no published independent factor analyses conducted in ASD or broader developmental disability samples and no independent estimates of screening effectiveness or clinical discriminant validity of the GARS-3. Therefore, the present project seeks to add to the limited research regarding the GARS-3 using program evaluation data from a large special education agency in Western New York state. The project consisted of three different studies that addressed aspects of GARS-3 internal structure validity and clinical discriminant validity. Study one involved an exploratory factor analysis (EFA) of the GARS-3 items with an ASD sample (n = 204) rated by special education teaching staff. Study two, confirmatory factor analyses (CFA) using a second ASD and non-ASD developmental disabilities (DDs) sample (n = 200), were used to examine the model fit of the published GARS-3 model and the factor model derived from the study one EFA, and assess which of the two models better fit the sample covariance matrix. Finally, aspects of the GARS-3's clinical discriminant validity were assessed using unique ASD cases from studies one and two (ASD sample n = 226) and an additional non-ASD developmental disabilities sample (non-ASD DDs sample n = 64) from the same special education agency. Clinical discriminant validity was examined via between-group comparisons, classification accuracy of a predetermined cut score, and exploration of other possible cut scores using receiver operator characteristic (ROC) curve analyses. The EFA resulted in a six-factor solution that was very similar in structure to the GARS-3 published six-factor model - differing only in the placement of one item. The CFAs indicated that the GARS-3 published model and the EFA-derived model both fit the data well and did not substantively differ. However, when cross-loadings were added, based on EFA results, CFA model fit significantly improved. ROC curve analyses indicated that, when using the suggested cut score of 70, sensitivity and specificity were lower than predicted. Lower cut scores yielded good sensitivity but poorer specificity, while higher cut scores showed the opposite pattern. Discussion and recommendations pertained to examining items and subscales based on cross-loadings and inter-factor correlations in addition to clinical implications of sensitivity and specificity findings.
Read
