Implementation strategies improve the use of evidence-based practices in routine healthcare settings. Little is known about the mechanisms by which educational implementation strategies like group consultation function. The first step in understanding strategy-mechanism linkages is to deconstruct multifaceted implementation strategies and identify which discrete components can be considered “active ingredients” because they are related to distal implementation outcomes of interest (e.g., fidelity). This study sought to demonstrate which of 3 group consultation components (i.e., case support via problem-solving discussions, skill rehearsal via role play, and feedback on videotaped sessions) were associated with improvements in clinician fidelity (i.e., manual adherence and competence) to an evidence-based parent-mediated naturalistic developmental behavioral intervention (NDBI) called Project ImPACT. We used a component analysis design. This study took place within an under-resourced community mental health system. Twenty clinicians across 6 agencies submitted weekly surveys and videotapes of telehealth parent coaching sessions with 21 families of Medicaid-enrolled autistic children during a baseline period of 3-9 weeks, and while receiving 12 weeks of group consultation in groups of 3-5 clinicians per agency. Consultation components were delivered in isolation in three, 4-week blocks. We used two methodologies to examine the extent to which discrete consultation components affected clinician fidelity: visual analysis and multilevel modeling. Coders masked to condition and timepoint used the Project ImPACT Coaching Fidelity Checklist to evaluate manual adherence and the PEACE measure to evaluate parent coaching competency. Analyses included 154 videotaped telehealth Project ImPACT sessions from the 12 clinicians at 3 agencies that had sufficient baselines. Average baseline manual adherence across clinicians was 62.67%, well below the fidelity threshold (80%). Manual adherence did not change from baseline. However, competency scores for reflective coaching practices (e.g., problem-solving obstacles to caregivers using NDBI strategies) in the Feedback consultation phase were 0.32 points higher than those in the Baseline phase (5-point Likert scale); this was a statistically significant increase, t(62.12) = 2.65, p = .01. No other consultation components were associated with changes in competency domains. In terms of clinician perspectives on each consultation component, there were statistically significant differences in ratings of the usability of each component; Skill Rehearsal was rated significantly lower than Case Support and Feedback. We were also interested in the effect of the group consultation model on other implementation outcomes. Case penetration (i.e., the proportion of eligible clients on a clinician’s caseload who received the intervention) did not significantly change from week to week in the baseline period, but did significantly increase in the consultation period – this was true both for clients using Medicaid insurance and for those using private insurance or self-pay. Contrary to hypothesis, positive perceptions of Project ImPACT increased in the baseline period, but not in the consultation period. This may have been due to a ceiling effect. We also found statistically significant improvements in most parent-reported social communication outcomes for the autistic children enrolled in the trial, despite the fact that families received the intervention from novice clinicians at low fidelity. Additional implementation support (e.g., leadership interventions) and a higher dosage of effective group consultation components (i.e., more than 6 hours across 4 weeks) are likely needed to improve clinician fidelity in under-resourced settings. Implications for future research on implementation strategy mechanisms and studies of group consultation are discussed.
Read
