115?:
‘3!- .

. .1- .1).
9.14“.” +3.3

.3...1\

Ti.
:28

:99

.
L

.w.

 

m-qm

l:

W

an“ mumnygmyg

:0“:

‘P

 

_1Mfl"“‘9‘1‘”0‘

uv-«n

   

' 5"!” 1" 'h MQF'WH

1. t

. 1...». my a...
.2 p "My“..‘ig

21. .A

 

M." MI

 

LIBRi‘RY

 

’i
”DIS Michigan State
‘ University

 

 

This is to certify that the
dissertation entitled

APPLYING DOSE-RESPONSE THEORY TO
COALITION TRAINING: UNDERSTANDING THE
RELATIONSHIP BETWEEN TRAINING DOSAGE AND
COALITION CAPACITY FOR COMMUNITY PROBLEM
SOLVING FOR SUBSTANCE ABUSE PREVENTION

presented by
Evelyn Yang

has been accepted towards fulfillment
of the requirements for the

Doctoral degree in Psychology

flMvvz/ULL add ’11 9i ‘3 LVonr *\

' Major Professor’s SignatiJre

II JZOJO?

Date

MSU is an Affirmative Action/Equal Opportunity Employer

 

 

 

PLACE IN RETURN BOX to remove this checkout from your record.
TO AVOID FINES return on or before date due.
MAY BE RECALLED with earlier due date if requested.

 

DATE DUE

DATE DUE DATE DUE

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

5/08 KzlProj/AocaPrelelRC/Dateoue indd

APPLYING DOSE-RESPONSE THEORY TO COALITION TRAINING:
UNDERSTANDING THE RELATIONSHIP BETWEEN TRAINING DOSAGE AND
COALITION CAPACITY FOR COMMUNITY PROBLEM SOLVING FOR
SUBSTANCE ABUSE PREVENTION

By

Evelyn Yang

A DISSERTATION
Submitted to
Michigan State University

in partial fulfillment of the requirements
for the degree of

DOCTOR OF PHILOSOPHY
Psychology

2009

ABSTRACT
APPLYING DOSE-RESPONSE THEORY TO COALITION TRAINING:
UNDERSTANDING THE RELATIONSHIP BETWEEN TRAINING DOSAGE AND
COALITION CAPACITY FOR COMMUNITY PROBLEM SOLVING FOR
SUBSTANCE ABUSE PREVENTION
By
Evelyn Yang

Coalitions are common vehicles to reduce substance abuse on a population-wide
scale. Coalitions bring together diverse community representatives to work
collaboratively toward achieving a common goal, and the level of skill, capacity and
effort required for coalitions to be effective is quite substantial. Current emphasis is on
building coalitions to be strong community problem solvers, i.e., local experts with the
tools, knowledge and expertise to effectively address their own community concerns.
Training has been used to help coalitions develop these skills and knowledge to be
successful. However, one of the limitations with existing training studies is lack of
information on how much training is needed to benefit the coalition. Dose-response
modeling has been used in other fields to understand the relationship between
intervention amount and intervention outcomes.

This study applied dose-response modeling to coalition training provided by a
national training institute to increase understanding of the conditions under which
training can increase a coalition’s community problem solving capacity to successfully
prevent substance abuse. A pre- and post-test study design with a control condition was
used to examine the relationship between training and three elements of community

problem solving capacity: coalition internal functioning, comprehensiveness of strategies,

and facilitation of community changes. The following research questions were examined

in this study:
1. Does training improve coalition capacity for community problem solving?
2. What is the nature of the relationship between training dosage and elements of
capacity for community problem solving?
3. Which type of training dosage received is the strongest predictor of each element
of capacity for community problem solving?

This study found that coalitions that have received training have higher post-test
levels of coalition internal functioning than those that have not been trained. Study
findings indicate a weak relationship between training dosage and elements of capacity
for community problem solving. High levels of training frequency, operationalized as the
number of individuals attending more than one training, are associated with greater
coalition internal functioning. The relationship between training and community problem
solving capacity appears to be complex. Other factors including receipt of technical
assistance and transfer of learning from training and technical assistance appear to be

critical elements needed to build a coalition’s capacity for community problem solving.

ACKNOWLEGEMENTS

I would first of all like to acknowledge my dissertation committee, Dr. Pennie
Foster-Fishman (Chair), Dr. William Davidson, Dr. Kevin Ford and Dr. Ryan Bowles. I
thank them for their time and effort to guide me through the dissertation process and for
their critical and thoughtful feedback. Dr. Foster-Fishman, especially, has provided a
great deal of support to me, not just as my dissertation chair, but also as my advisor.
Thank you for encouraging me to finish my degree and for working with me this past
year even while you were on sabbatical. l have learned a great deal under your tutelage
and guidance, making me a stronger community psychologist.

I also want to thank my mentor, Dr. James Cook. Thank you for inspiring in me
an interest in systems reform and coalition work. You have had faith in me ever since I
was your student, and I thank you for providing support to me through this process.
Thank you for reading drafts, providing helpful feedback, and for all your
encouragement.

This study could not have been conducted without the amazing efforts conducted
by CADCA and its National Coalition Institute. It is only through the contributions this
organization and its staff have made to the substance abuse field that this study is
possible. I also would like to acknowledge the staff at CADCA. Thank you, General
Dean, for the support you have given me to complete my degree. I truly appreciate the
flexibility you have provided. To Jane Callahan, thank you for reviewing drafis, your
encouragement, your faith me in, and your flexibility and understanding throughout this

process. Additionally, thank you so much to the entire Institute staff — Kareemah

Abdullah, Eduardo Hernandez, Andrea de la Flor, Anne Fitzgerald, Sue Stine, Carlton
Hall, Rachael Seda, Nancy Roy, Janine Payne, Kristine von Brook, Kristy Miller and
Kyle Robertson. I love working with you all, and I truly appreciate all your kind words
of support and encouragement.

Finally, I would like to thank my family and friends. Thank you to my parents for
their prayers, love and constant support. They have always encouraged me to achieve
my goals and have provided much needed advice and support to help me complete my
degree. To my sister, Carolyn, thank you for taking care of me, walking the dog and
making sure I ate while I was in my “dissertation mode.” Christy, thanks for the check-
ins and encouragement, even all the way from Korea. I thank my grandparents in
Houston and Seoul for their prayers and encouragement. And finally, thank you to all my
fi'iends, near and far, who have journeyed with me through this process. To all my family
and friends - whether it was statistical advice, words to encourage me or acting as a
sounding board to process my work, you have all been an immense support system to me

and I am truly grateful for having such wonderful people in my life. Thank you.

ii

TABLE OF CONTENTS

LIST OF TABLES ............................................................................
LIST OF FIGURES ...........................................................................

CHAPTER I
INTRODUCTION .............................................................................

CHAPTER 2
LITERATURE REVIEW .....................................................................
Coalitions as Mechanisms to Prevent Substance Abuse .........................
Coalitions as Community Change Agents for Population-Level
Outcomes ..............................................................................
Community Problem Solving for Substance Abuse Prevention .................
Coalition lntemal Functioning .............................................
Comprehensiveness of
Strategies .....................................................................
Facilitation of Community
Changes ........................................................................
The Study Context .....................................................................
Relationship between Training and Coalition Community Problem
Solving. ................................................................................
Dose-Response Theory ...............................................................
Defining Training Dosage ...................................................
Examining Multiple Measures of Dose ...................................
Calculating Coalition Training Dosage ....................................
Modeling the Dose-Response Relationship .......................................
Linear Model ..................................................................
Curvilinear Model ............................................................
Threshold Model .............................................................
Conclusion ....................................................................
Coalition Age as a Moderator of Training Dosage and Coalition
Outcomes .....................................................................
Additional Considerations in Dose-Response Modeling —
Controlling for Covariates ..................................................
Current Study ..........................................................................

CHAPTER 3
RESEARCH DESIGN AND METHODS ..................................................
Overview ...............................................................................
Procedures ..............................................................................
Setting ........................................................................
Sample ........................................................................

iii

vi

xi

ll
11

15
18
21

28

31
35

40
43
46
54
56
60
61
63
66
67

68

69
70

75
75
75
75
76

Power to Detect Differences ...............................................

Measurement Development and Data Collection .................................

Training Dosage ..............................................................
Coalition Capacity for Community Problem Solving ...................

Measures ...............................................................................

CHAPTER 4

RESULTS . . ..

Training Dosage ..............................................................
Coalition Capacity for Community Problem Solving ...................

Outliers .................................................................................
Univariate and Descriptive Analyses ...............................................
Missing Data Imputation .............................................................
Outcome Measure Scale Construction .............................................

Internal Functioning .........................................................
Comprehensiveness of Strategies ..........................................
Facilitation of Community Changes .......................................

Survey Year ............................................................................
Improving Statistical Power .........................................................

Coalition Internal Functioning .............................................
Comprehensiveness of Strategies ..........................................
Facilitation of Community Change .......................................

Results of Hypothesis Testing ......................................................

Research Question 1: Does training improve a coalition’s
community problem solving capacity? ...................................
Research Question 2: What is the nature of the relationship between
training dosage and elements of capacity for community problem
solving? .......................................................................
Research Question 3: Which type of dosage is the strongest
predictor of each element of capacity for community problem
solving? .......................................................................
Proposition #1: Coalitions with both high training breadth and
frequency will have the highest levels of improvement in each of
the three elements of community problem solving capacity ............
Proposition #2: The amount of training a coalition needs to build
coalition capacity for community problem solving will be related to
the age of the coalition .......................................................

Further Examination of Other Factors Related to Community Problem
Solving Capacity .....................................................................

Technical Assistance and its Relationship to Coalition Community

Problem Solving ..............................................................
Overall CADCA Touch .....................................................
Transfer of Learning .........................................................
Transfer of Learning by Varying Levels of CADCA Touch ...........
Testing CADCA’s Theory of Change .....................................

78
79
79
80
83
83
84

95
95
95
96
98
99
100
101
102
106
106
107
108
109

109

114

117

126

131

141

141
144
150
154
155

CHAPTER 5

DISCUSSION .................................................................................. 158
Key Findings ........................................................................... 158
Training Impacts on Coalition Capacity for Community Problem
Solving ........................................................................ 159
Dose-Response Relationship for Training and Community Problem
Solving Capacity ........................................................... 163
Technical Assistance and Community Problem Solving Capacity 167
CADCA Touch and Capacity for Community Problem
Solving ........................................................................ 168
Implications for Coalition Training and Technical Assistance .................. 171
Study Strength and Limitations ...................................................... 176
Conclusions and Recommendations for Future Research ..................... 177
APPENDICES ................................................................................. 181
REFERENCES ................................................................................. 270

LIST OF TABLES

Table 1. Description of CADCA Institute Trainings to Build Coalition Capacity
for Community Problem Solving ............................................................

Table 2. Summary of Institute Trainings .....................................................
Table 3. Examples of CADCA Trainings ...................................................
Table 4. Sample Demographic Characteristics .............................................
Table 5. Descriptive Statistics on Training Dosage .......................................
Table 6. Correlations between Dosage Types .............................................

Table 7. Bivariate Relationships of Demographic Characteristics with Missing
Data Status ......................................................................................

Table 8. Bivariate Relationships of Demographic Characteristics with Baseline
Survey Year ......................................................................................

Table 9. Covariates Predicting Time 2 Coalition Internal Functioning ................
Table 10. Covariates Predicting Time 2 Comprehensiveness of Strategies ............
Table 11. Covariates Predicting Time 2 Facilitation of Change ........................

Table 12. Bivariate Relationships of Training Status with Time 1 Elements of
Community Problem Solving Capacity .....................................................

Table 13. Training Status Predicting Coalition Internal Functioning ...................
Table 14. Training Status Predicting Comprehensiveness of Strategies ...............
Table 15. Training Status Predicting Facilitation of Community Change..........

Table 16. Coalition Internal Functioning and Training Dosage: Curve Fit Model
Summary and Parameter Estimates ...........................................................

Table 17. Comprehensiveness of Strategies and Training Dosage: Curve Fit Model
Summary and Parameter Estimates ..........................................................

Table 18. Facilitation of Change and Training Dosage: Curve Fit Model Summary
and Parameter Estimates ......................................................................

vi

36

37

38

78

96

96

97

102

107

108

109

110

111

112

113

115

116

116

Table 19. Correlation Matrix of IVs and DVs (Dosage Sub-Sample) ..................
Table 20. Training Dosage Predicting Coalition lntemal Functioning ..................
Table 21. Training Dosage Predicting Comprehensiveness of Strategies .............

Table 22. Training Dosage Predicting Facilitation of Community
Change ............................................................................................

Table 23. Training Breadth Quartiles Predicting Coalition lntemal Functioning
(n=119) ............................................................................................

Table 24. Training Frequency Quartiles Predicting Coalition lntemal Functioning
(n=131) ............................................................................................

Table 25. Training Intensity Quartiles Predicting Coalition lntemal Functioning
(n=137) ...........................................................................................

Table 26. Training Breadth Quartiles Predicting Comprehensiveness of Strategies
(n=119) ..........................................................................................

Table 27. Training Frequency Quartiles Predicting Comprehensiveness of
Strategies (n=131) ..............................................................................

Table 28. Training Intensity Quartiles Predicting Comprehensiveness of Strategies
(n=137) ............................................................................................

Table 29. Training Breadth Quartiles Predicting Facilitation of Community
Change (n=119) .................................................................................

Table 30. Training Frequency Quartiles Predicting Facilitation of Community
Change (n=131) .................................................................................

Table 31. Training Intensity Quartiles Predicting Facilitation of Community
Change (n=137) .................................................................................

Table 32. Interaction of Training Breadth and Frequency Predicting Coalition
lntemal Functioning .............................................................................

Table 33. Interaction of Training Breadth and Frequency Predicting
Comprehensiveness of Strategies ..............................................................

Table 34. Interaction of Training Breadth and Frequency Predicting Facilitation of
Community Change .............................................................................

vii

117

119

119

120

122

122

123

123

124

124

125

125

126

128

129

130

Table 35. Coalition Age as Moderator of Relationship between Training Breadth
and Coalition lntemal Functioning ............................................................ 132

Table 36. Coalition Age as Moderator of Relationship between Training
Frequency and Coalition lntemal Functioning .............................................. 133

Table 37. Coalition Age as Moderator of Relationship between Training Intensity
and Coalition lntemal Functioning ............................................................ 134

Table 38. Coalition Age as Moderator of Relationship between Training Breadth
and Comprehensiveness of Strategies ......................................................... 135

Table 39. Coalition Age as Moderator of Relationship between Training
Frequency and Comprehensiveness of Strategies ........................................... 136

Table 40. Coalition Age as Moderator of Relationship between Training Intensity
and Comprehensiveness of Strategies ......................................................... 137

Table 41. Coalition Age as Moderator of Relationship between Training Breadth
and Facilitation of Community Change ...................................................... 138

Table 42. Coalition Age as Moderator of Relationship between Training
Frequency and Facilitation of Community Change ....................................... 139

Table 43. Coalition Age as Moderator of Relationship between Training Intensity
and Facilitation of Community Change ...................................................... 140

Table 44. Bivariate Relationships of Technical Assistance Status with Elements of
Community Problem Solving Capacity ...................................................... 141

Table 45. Technical Assistance Predicting Coalition lntemal
Functioning ....................................................................................... 143

Table 46. Technical Assistance Predicting Comprehensiveness of

Strategies .......................................................................................... 143
Table 47. Technical Assistance Predicting Facilitation of Community

Change ............................................................................................ 144
Table 48. Means and SD for Community Problem Solving by CADCA

Touch .............................................................................................. 145
Table 49. ANOVA Table for CADCA Touch on Pre-Test Community Problem

Solving ............................................................................................ 146
Table 50. CADCA Touch Predicting Coalition lntemal Functioning ................... 149

viii

Table 51. CADCA Touch Predicting Comprehensiveness of
Strategies .........................................................................................

Table 52. CADCA Touch Predicting Facilitation of Community
Change ............................................................................................

Table 53. Transfer of Learning Predicting Coalition lntemal
Functioning ......................................................................................

Table 54. Transfer of Learning Predicting Comprehensiveness of

Strategies ..........................................................................................

Table 55. Transfer of Learning Predicting Facilitation of Community
Change ............................................................................................

Table 56. Change in lntemal Functioning Predicting Comprehensiveness of
Strategies .........................................................................................

Table 57. Change in Comprehensiveness of Strategies Facilitation of Community
Change

Table A. Variable Comparison for Time 1 and Time 2
Table B. Descriptive Statistics for Categorical Variables (Total Sample)
Table C. Descriptive Statistics of Scales and Continuous Items (Total Sample)

Table D. Correlation Matrix of IVs, Transfer of Learning and DVs (Dosage Sub-
Sample)

Table E. Total Variance Explained for Time 1 Coalition lntemal Functioning
Principal Component Analysis

Table F. Total Variance Explained for Time 2 Coalition lntemal Functioning
Principal Component Analysis

Table G. Total Variance Explained for Time 1 Comprehensiveness of Strategies
Principal Component Analysis

Table H. Total Variance Explained for Time 2 Comprehensiveness of Strategies
Principal Component Analysis

Table 1. Total Variance Explained for Time 2 Facilitation of Community Change
Principal Component Analysis

149

150

153

154

154

156

157

181

183

184

186

187

187

188

188

188

Table]. Component Matrix for Time 2 Coalition lntemal Functioning Principal
Component Analysis 189

Table K. Component Matrix for Time 2 Facilitation of Community Change
Principal Component Analysis 190

LIST OF FIGURES

Figure 1. CADCA Institute’s Framework for Effective Community Problem
Solving ............................................................................................ 6

Figure 2. CADCA Institute’s Training Impact of Framework for Effective
Community Problem Solving ................................................................. 20

Figure 3. Dose-Response Relationship for Training Dosage and Coalition lntemal
Functioning ..................................................................................... 71

Figure 4. Dose-Response Relationship for Training Dosage and
Comprehensiveness of Strategies ............................................................ 71

Figure 5. Dose-Response Relationship for Training Dosage and Facilitation of

Community Change ............................................................................ 71
Figure 6. Training Frequency by Survey Year ............................................ 105
Figure 7. Time 2 Coalition lntemal Functioning by Survey Year ...................... 105

Figure 8. Coalition lntemal Functioning by Data Collection Period and Training
Status ............................................................................................. 114

Figure 9. Time 2 Coalition lntemal Functioning by Survey Period and CADCA

Touch ............................................................................................ 147
Figure A. Scatterplot of Training Breadth and Coalition lntemal Functioning ........ 191
Figure B. Scatterplot of Training Frequency and Coalition lntemal Functioning. . 192
Figure C. Scatterplot of Training Intensity and Coalition lntemal Functioning ....... 193
Figure D. Scatterplot of Training Breadth and Comprehensiveness of Strategies. 194
Figure B. Scatterplot of Training Frequency and Comprehensiveness of

Strategies ........................................................................................ 195
Figure F. Scatterplot of Training Intensity and Comprehensiveness of Strategies. . .. 196

Figure G. Scatterplot of Training Breadth and Facilitation of Community
Change ........................................................................................... 197

Figure H. Scatterplot of Training Frequency and Facilitation of Community
Change ........................................................................................... 198

Figure I. Scatterplot of Training Intensity and Facilitation of Community
Change ........................................................................................... 199

xii

CHAPTER 1: INTRODUCTION

Coalitions are collaborative entities comprised of various stakeholders, agencies
and community groups that work together to address a common goal of improving the
health and well-being of its community members and environment. Coalitions are
commonly formed for public health and promotion to address some of the most
challenging problems facing society such as teen pregnancy (Paine-Andrews et a1., 2002),
cardiovascular disease and substance abuse (Francisco, Paine, & F awcett, 1993),
substance abuse prevention (Hallfors, Cho, Livert, Kadushin, 2002; Hingson, Zakocs,
Winter, Rosenbloom, & DeJong, 2005), and child immunization (Butterfoss, 2007). The
fundamental premise of coalition theory argues that this country’s most difficult
problems are unlikely to be solved by a single agency or organization and that a
collaborative response is needed (Lasker & Weiss, 2003).

Complex community health problems require comprehensive, collaborative
solutions in order to achieve benefit for the entire community or targeted population.
This broad degree of change, called population-level change, is the ultimate goal of
coalitions (Roussos & Fawcett, 2000). Population-level outcomes represent changes in
health status, behavior changes in a population sample and improvements in social
indicators of well-being. Examples of population-level improvements in health outcomes
include reductions in rates of domestic violence, decreases in teen pregnancy rates, and
reductions in the incidence and prevalence of substance abuse among youth.

However, population-level outcomes are difficult to achieve. Evaluations of
coalition efforts have shown only a limited impact in modifying behaviors at the

population level (Fawcett et al., 2001; Merzel & D’Affitti, 2003; Roussos & F awcett,

2000). Other studies have demonstrated nonsignificant findings, such as the national
evaluation of the Robert Wood Johnson Foundation’s Fighting Back Initiative, a multi-
million dollar coalition initiative targeting substance abuse prevention. The evaluation
found that participating coalitions were unsuccessful at reducing population-level
substance abuse rates (Hallfors, Cho, Livert, Kadushin, 2002). Many comprehensive
reviews of coalition efforts have concluded “only limited empirical evidence exists on
their effectiveness in improving community-level outcomes” (Roussos & F awcett, 2000,
p. 370). While coalition success stories do exist, they tend to be found in individual case
studies, such as a Midwest coalition’s contribution to reductions in teen pregnancy rates
(Paine-Andrews et al., 2002) and declines in infant mortality rates in an initiative
targeting a high-risk area of Boston (Plough & Olafson, 1994 as cited in Kreuter, Lezin &
Young, 2000). While promising, these study findings are limited in their generalizability
to other settings and issues.

The difficulty in linking coalitions to population-level outcomes has many
proposing the need to examine other more proximal outcomes signifying a coalition’s
success. One of the criticisms made by coalition researchers (Allen, Watt, & Hess, 2008;
Green & Kreuter, 2002; Roussos & F awcett, 2000) is that much of the previous coalition
research has focused on the direct effect of coalitions on population-level change
variables, such as the Fighting Back study. This has a variety of researchers (Kreuter et
al., 2000; Merzel & D’Affitti, 2003; Roussos & Fawcett, 2000) suggesting that one of the
reasons for the modest findings of coalition evaluations may be that improvements in
widespread behaviors and community health outcomes are so far downstream that they

are unlikely to be measured within the timeframe of most evaluation studies.

Consequently, efforts need to be made to determine intermediate outcomes that are more
likely to be measured within the timeframe of most evaluations and that coalitions can
affect more directly. By identifying and assessing relevant intermediate outcomes,
success for the coalition field may be redefined in such a way to offer more realistic and
appropriate targets to achieve within a fairly short time frame. Additionally, an improved
focus on coalition intermediate outcomes may also provide a stronger understanding of
the process by which coalitions achieve population-level change (Roussos & F awcett,
2000)

One core intermediate outcome proposed by Roussos and F awcett (2000) is
community change, which are new or modified programs, policies and practices brought
about by the coalition related to its mission. Examples of community changes include:
infusing contraceptive information into abstinence only pregnancy prevention
programming, expanding health clinic hours so that community members have better
access to these services, and changing policy to increase the tax paid for tobacco
products. These community changes represent progress toward achieving long-term goals
and can be documented with reliability (Fawcett et al., 1997; Francisco, Paine, &

F awcett, 1993). Fawcett and colleagues have demonstrated this link in a variety of
empirical case studies, including those of teen pregnancy prevention and substance abuse
prevention coalition efforts. These cases documented the relationship between high
levels of community change and subsequent reductions in the targeted public health
problems (Fawcett et al., 1997; Paine-Andrews et al., 2002). Other researchers agree that
markers of intermediate outcomes toward more distal health outcomes are changes in the

community environment, such as the shifts in programs, policies and practices described

above (Butterfoss, 2007; Kegler, Twiss & Look, 2000). Given the time frame, it was not
possible to see population-level outcomes within the scope of this particular study.
However, as community change agents, coalitions are helping to bring about new and
modified programs, polices and practices that address their mission. Since promise exists
in the role of coalitions as community change agents, further study is needed to determine
how coalitions can become the change agents that facilitate improvements in these
intermediate outcomes. This study examines the role training plays to help coalitions
accomplish these intermediate outcomes (i.e., community changes) indicative of coalition
effectiveness. Specifically, this study investigates how training impacts a coalition’s
ability to facilitate these changes, and whether higher levels of training participation are
associated with greater degree of community changes facilitated by the coalition.
Research examining the relationship between community changes and population-
level outcomes provides some indication that these community changes are important
precursors of improvements in population-level health outcomes. F awcett and colleagues
(1997; Paine-Andrews et al., 2002) have demonstrated in numerous case studies that
reductions in population-level health outcomes are more likely when a sufficient number
of community changes have been achieved. When coalitions are effective at facilitating
new or modified programs, policies and practices, such as increasing enforcement of
underage drinking laws, increasing availability of contraceptives, and implementing
evidence-based prevention programs, they are more likely to experience population-level
decreases in targeted health outcomes, such as the reduction of teen pregnancy rates and
alcohol-related crashes and fatalities. These studies provide strong examples of

community changes as intermediate outcomes toward population-level outcomes.

 

However, it is apparent that not all coalitions are successful at facilitating
community changes. Kreuter and colleagues (2000) examined 68 studies of coalition
efforts to change health systems and health status. Of these, only two documented
examples were found where coalitions were likely to have contributed to systems
improvements, such as the establishment of a clinic in a health services initiative for rural
homeless and changes in blood lead testing practices in a lead poisoning initiative.
Hingson and colleagues (2005) reexamined the Fighting Back coalitions in a post hoc
study, studying more closely the number of community changes these coalitions helped
bring about to reduce availability of alcohol and expand access to alcohol treatment. Of
the 12 Fighting Back communities included in this study, the majority were not
successful at changing their community systems at high enough levels to facilitate
population-level reductions in alcohol related crash fatalities.

It is critical, then, to think about how to help coalitions become active community
change agents, engaged in making concerted modifications of programs, policies and
practices in their communities. One approach taken by a large, national training
organization is a community problem solving approach. Community Anti-Drug
Coalitions of America (CADCA) is a national, nonprofit organization whose mission is
“to strengthen the capacity of community coalitions to create and maintain safe, healthy
and drug-free communities.” CADCA houses the National Coalition Institute (Institute)
that has the aim of building more effective community anti-drug coalitions through
training, technical assistance and education. Since its inception in 2002, the Institute has
trained over 2800 coalitions and other substance abuse prevention groups using a

community problem solving for change framework based upon a model developed by the

University of Kansas (Collie-Akers, F awcett, Schultz, Carson, Cyprus & Pierle, 2007;
Fawcett et al., 2001; Roussos & F awcett, 2000). This community problem solving
approach puts solution-generation in the hands of the community who are the real experts
on their local issues and the resources they can put into play to address these concerns
(Lasker & Weiss, 2003). The Institute’s community problem solving approach to
substance abuse prevention encompasses a comprehensive theory of change that starts
with building internal coalition functioning and ends with population-level reductions in
substance abuse rates (see Figure 1).

Figure I. CADCA Institute’s Framework for Effective Community Problem Solving

 

 

 

 

 

 

 

Strong Coalitions
Coalition Pursuing Facilitating Population
Internal Comprehensive Community " Level
Functioning Strategies Changes Changes

 

 

 

 

 

 

 

 

 

 

 

 

The intent of CADCA Institute training is to improve coalition community
problem solving capacity in the following areas: improving coalition internal functioning,
helping coalitions pursue comprehensive strategies and activities, and helping them
facilitate community changes, which then contribute to population-level changes in
health outcomes. The Fighting Back study (Hallfors, et al., 2002) clearly indicates that
the mere existence of a coalition is unlikely to achieve outcomes. Current research
suggests effective coalitions attune to improving both internal (e.g., coalition operations,
assessment and planning) and external functioning (e. g., interventions and strategies)
(Foster-Fishman, Berkowitz, Lounsbury, Jacobson & Allen, 2001; Stevenson & Mitchell,

2003; Zakocs & Edwards, 2006; Zakocs & Guckenburg, 2007). This model incorporates

both aspects of functioning, also linking them to coalition engagement as community
change agents, an intermediate outcome toward population-level change.

However, becoming a coalition capable of strong community problem solving is
difficult, and efforts must be made to help develop coalitions develop this capacity.
Training is one important tool that has often been used to support coalitions in their
development and success (Butterfoss, Morrow, Webster & Crews, 2003; Florin, Mitchell
& Stevenson, 1993; Quinby et al., 2008). Coalition training is associated with improved
knowledge and skills, such as understanding community risk and protective factors
(Feinberg, Greenberg, Osgood, Anderson & Babinski, 2002) and coalition building and
public policy action (Stevens & Lodl, 1999). Training is recommended as a critical tool
to support coalitions in becoming active community change agents (Roussos & F awcett,
2000) and appears to be an essential tool for building coalition capacity for community
problem solving. .

While training appears to be necessary and is occurring on a frequent basis, the
coalition field still lacks information about the correct form and function of training
needed to achieve optimal results. Investments in training are costly, requiring both
monetary and human investments. Additionally, extensive research on training
specifically for coalitions has not been conducted. How training is offered and the extent
to which coalitions take advantage of training varies considerably. However, more
detailed, specific information is needed on what amounts of training participation are
critical for coalitions to benefit from training. This study focused on determining the
relationship between different amounts of training and different types of coalition

training outcomes in order to better determine how training can benefit coalitions and

how coalitions can use scare resources to make more informed decisions about how they
access training.

Other fields, such as pharmacology (Calabrese and Baldwin, 2001) and
psychotherapy (Howard, Kopta, Krause, & Orlinsky, 1986) have looked to dose-response
theory to understand if higher amounts of an intervention, such as training, are related to
better outcomes. Dose is a measure of exposure or participation in an activity or
intervention, such as attendance in a youth-focused program to prevent teen pregnancy,
exposure to a media campaign to reduce alcohol and drug abuse, and attendance at a
training event. The link between dose and outcomes has been tested in other fields,
suggesting that higher dose is associated with better outcomes; for example, greater
involvement in an after-school program is related to better grades (Chaput, Little &
Weiss, 2004; Marsh & Kleitman, 2002).

However, dose-response theory has not been applied comprehensively to coalition
training. While coalitions are often trained as part of grant requirements (Department of
Health and Human Services RFA No. SP-09-002), we do not understand how
participation in training should be operationalized. Does greater attendance, as measured
by a greater number of individuals trained within a coalition, lead to greater coalition
capacity for community problem solving? If coalitions send a team of members to a
training, does the coalition reap more benefits than one that sends just a single member?
As of yet, research has not provided answers to these questions. There is little empirical
evidence to enhance understanding of how training dosage should be conceptualized, nor
how varying levels of training dosage relate to coalition outcomes. This study undertook

the first large-scale approach to understanding this relationship and developing a

framework to understand coalition training dosage. Specifically, this study focused on
determining the measures of training dosage that are relevant for coalitions and the
relationship between different types of dosage and a coalition’s capacity for community
problem solving.

This study focused on the role one major training institute plays in helping
coalitions become community problem solvers for population-level outcomes. The
CADCA Institute trains on each aspect of the community solving framework described in
Figure 1, and its training is designed to help coalitions improve their capacity in each
component of the model. Evaluations of the Institute’s model suggest that coalitions
trained by the Institute are more likely to have better internal functioning, be more
comprehensive and engage as community change agents (F oster-Fishman, Law & Ahn,
2008). The relationship between Institute training and coalition achievement of distal,
population-level outcomes has not yet been tested. However, the Institute’s evaluation
findings support the training center’s role in building coalition capacity to achieve
community changes, the intermediate outcome toward population-level outcomes. This
study continued to investigate Institute training and its relationship to building coalition
community problem solving capacity, and attempted to fill a gap in the training literature
by examining more closely how varying levels of training participation relate to different
community problem solving capacity outcomes.

In conclusion, the purpose of this study was to understand how training increases
coalition capacity for community problem solving for population-level substance abuse
prevention by examining the role training participation plays to improve coalition skills

and capacities in different elements of community problem solving. This study examined

how training builds coalition skills and investigated more closely the specific relationship
between training attendance amount and coalition capacity for community problem
solving. If training can build strong coalition community problem solvers for population-
level substance abuse prevention and aspects of training attendance necessary for training
to be successful are defined, then future studies that examine the entirety of the theory of
change depicted in Figure 1 can be carried out (i.e., how community change triggers
long-term benefits in improved behaviors and health indicators). This is necessary to
fully understand how coalitions can be effective vehicles for population-level change and
the key supports needed to build their capacity to do so.
To explicate the relationship between training exposure and coalition community

problem solving capacity, the following research questions were examined in this study:

1. Does training improve coalition capacity for community problem solving?

2. What is the nature of the relationship between training dosage and

elements of capacity for community problem solving?
3. Which type of training dosage received is the strongest predictor of each

element of capacity for community problem solving?

10

CHAPTER 2: LITERATURE REVIEW
Coalitions as Mechanisms to Prevent Substance Abuse
Substance abuse is a problem facing each community in America. The
pervasiveness of substance abuse is apparent in how it cuts across racial, socio-economic,
geographic and generational lines (Substance Abuse and Mental Health Services
Administration, Office of Applied Studies, 2008). Results from the 2008 Monitoring the
Future study, an annual survey on in the incidence and prevalence of youth substance use
rates funded by the National Institute on Drug Abuse (NIDA), indicate that past declines
in substance abuse rates are leveling off and researchers are concerned that softening
attitudes among youth on the perceived risk and disapproval of marijuana and inhalant
use may predict future increases in usage rates (Johnston, O'Malley, Bachman, &
Schulenberg, 2008). With 47.4% of 12th graders reporting they have used an illicit drug
in their lifetime, 32.1% of 8th graders reporting drinking alcohol in the last year, and
13.8% of 10th graders reporting using marijuana in the last month, the potential costs of
substance abuse are enormous, especially if efforts are not put into prevention and
treatment. These costs go beyond the individual and their family, affecting society as
well. According to the National Highway Traffic Safety Administration (2008), while
the proportion of alcohol-related traffic fatalities attributable to alcohol has been in
decline in the last 20 years, as of 2005 they were still as high as 33%. Additionally, the
cost of substance abuse to society is staggering; in 1992 the estimated economic cost of
drug and alcohol abuse was $245.7 billion. This includes prevention and treatment costs
as well as costs accrued in the justice and social welfare systems (Harwood, Fountain &

Liverrnore, 1999).

ll

Starting in the 19805, private foundations (Hallfors etal., 2002) and the federal
government (Yin and Kaftarian, 1997) began to put funds into developing local
community coalitions to address the substance abuse epidemic. The core belief behind
this approach was that substance abuse prevention required a collaborative, community-
based solution to change conditions in the community environment, system and structures
that contributed to substance abuse problems (Saxe et al., 1997). Coalitions are “a group
of individuals representing diverse organizations, factions or constituencies who agree to
work together in order to achieve a common goal” (F eighery & Rogers, 1989, p.1). Since
no two communities are alike, the premise of coalitions for substance abuse prevention is
that local, coordinated efforts best address the local, unique needs of the community.
Substance abuse coalitions are comprised of multiple groups of stakeholders, often a mix
of citizens, including youth and parents, service agencies, including prevention and
treatment providers, public health, law enforcement, schools, and community
organizations, including representatives from the faith community, businesses and
volunteer groups. By encouraging collaboration among stakeholders, these coalitions
take a comprehensive approach to addressing substance abuse in their communities.
Recent studies conducted on substance abuse coalitions suggest that most coalitions
appear to be serving rural areas with budgets of less than $200,000 (ONCDP, 2007;
Foster-Fishman et al., 2008). Coalitions vary in size from small to large (Butterfoss,
2007) and also in their focus, some focusing solely on substance abuse and others
including it as a priority among larger community health issues (Stevenson & Mitchell,
2003). Substance abuse coalitions are engaged in a variety of strategies and activities,

including implementing evidence-based prevention programs (Collins, Johnson &

12

Becker, 2007), providing information about the harms associated with substance abuse
(Florin et al., 1993), and changing local policies to enforce underage drinking laws
(Hingson et al., 2005).

Over time, coalitions have become very popular mechanisms for addressing
substance abuse at the community level (Collins et al., 2007; Florin et al., 1993; Green &
Kreuter, 2002; Hallfors, et a1, 2002; Hingson et al., 2005; Stevenson & Mitchell, 2003).
The Drug-Free Communities Support Program (DFC) has funded approximately 1500
coalitions since 1998 (Department of Health and Human Services RF A No. SP-09-002);
the Center for Substance Abuse Prevention will be investing over $38 million dollars to
fund an additional 20 states, territories and tribal entities through their Strategic
Prevention Framework-State Incentive Grants, which requires funding to be dispersed to
communities to implement local, strategic efforts to prevent substance abuse (Center for
Substance Abuse Prevention RF A No SP-09-011); the Community Capacity and
Development Office within the Office of Justice Programs funds Weed & Seed
communities, which are local coalitions engaged in law enforcement and community
mobilization efforts to address both the supply and demand for drugs (Department of
Justice, 2009). While the investment into substance abuse coalitions has been substantial,
it represents a small portion of the overall federal budget. Regardless, much attention has
been placed on coalitions as solutions to prevent substance abuse problems in this
country. Even with comparably limited resources in comparison to substance abuse
treatment and interdiction (Carnevale Associates, 2008), the spotlight on coalitions is
intense.

Consequently, the bar for coalition success is set high. Coalitions are considered

13

a “population strategy”, targeting an entire community to achieve the largest impact on
the public health issue (Merzel & D'Affitti, 2003). As such, accountability must be placed
at the population-level. While grant funding exists to create and maintain coalitions,
coalitions are being held to accountability measures for long-term reductions in substance
use and abuse within a population sample. For example, the Drug-Free Communities
Support Program (DF C) Grant administered by the White House Office of Drug Control
Policy has designated “core measures” that grantees are required to report on, which
include measures of past 30-day use and lifetime use of alcohol, tobacco and marijuana
among youth. Grantees are required to track and report data at the population-level for
the community they are targeting with their prevention efforts (Department of Health and
Human Services RFA No. SP-O9-002; ONDCP, 2007; ONDCP, 2008).

While there is a growing number of substance abuse coalitions, evaluations of
their ability to reduce substance abuse at the population-level are mixed. Perhaps one of
the most commonly cited coalition evaluations in the substance abuse field is that of the
Fighting Back Initiative, a 72 million dollar investment made by the Robert Wood
Johnson Foundation (Hallfors etal., 2002; Hingson et al., 2005; Zakocs & Guckenburg,
2006). Participating communities had a great deal of flexibility in determining how to
implement their grant and address substance abuse in their communities. The national
evaluation of the Fighting Back Initiative examined the specific intervention strategies
the coalitions implemented, the comprehensiveness of the strategies, and the coalition’s
success at reducing youth substance abuse. The researchers found that strategies
targeting youth and community prevention outcomes had no effect when compared to

matched community comparison sites. Additionally, some strategies targeting adults had

14

a negative effect in that adults in Fighting Back sites had worse program outcomes, such
as binge drinking, than adults in non-Fighting Back sites. Furthermore, communities that
implemented more comprehensive interventions were not more effective than those
communities who were less comprehensive (Hallfors et al., 2002). Comprehensiveness
was defined as implementing a broad array of strategies focusing on change at multiple
levels and multiple targets, such as use of youth prevention programs, reducing alcohol
outlet density and public awareness campaigns. Limited population-level change in
health behaviors have also been found in the Community Intervention Trial for Smoking
Cessation (COMMIT). While the intervention demonstrated decreases in smoking among
light to moderate smokers, no changes in heavy smokers were found, the primary target
group of the initiative (COMMIT Research Group, 1995).

Coalitions as Community Change Agents for Population-Level Outcomes

One potential reason for why research does not consistently show sufficient
evidence of coalitions influencing population-level health outcomes is that most studies
and funders have unrealistic expectations of the amount and type of outcome that can be
achieved within a funding or study period (Kreuter et al., 2000; Merzel & D’Affitti,
2003; Roussos & Fawcett, 2000). Population-level outcomes may take longer than the
lifetime of the grant funding period. For example, researchers in the COMMIT study
indicated that it might require more than the four years of the initiative to change
behaviors among heavy smokers (COMMIT Research Group, 1995). Given the
complexity of ameliorating a difficult community health issue such as substance abuse,
the current approach of focusing exclusively on long-term outcomes of changes in health

status and behaviors as the primary indicator of coalition effectiveness outcomes may

15

mask other important outcomes resulting from coalition work (Allen et al., 2008; Roussos
& Fawcett, 2000). Other more short-term and intermediate outcomes are ignored when
distal outcomes are the sole measure of coalition success. Allen and colleagues suggest
the need to put more attention and emphasis on the specific activities coalitions pursue to
achieve their mission. Their evaluation of domestic violence councils suggests that in
addition to sharing information and building relationships, these coalitions are effective at
promoting community changes to increase a community’s ability to respond to domestic
violence, such as modifying community practices or creating new programs. Focus on
community changes brought about by these entities appears to add value to understanding
coalitions (Allen et al., 2008).

Coalition researchers have made some progress by focusing on changes in the
community environment as intermediate outcomes toward population-level outcomes
(Allen et al., 2008; Collie-Akers et al., 2007; Hays, Hays, DeVille & Mulhall, 2000;
Kegler, Twiss & Look, 2000; Watson-Thompson, Fawcett, & Schultz, 2008). They
suggest that focusing on more near-term and intermediate markers of success provides
some benefit to coalition evaluations, improving traditional coalition evaluations that
have predominantly emphasized changes in distal health outcomes. These intermediate
outcomes consist of new and modified programs, policies and practices brought about by
the coalition related to its mission. Changing physician practice to incorporate drug and
alcohol prevention information during annual school physicals, improving lighting in
areas known for drug sales and illicit behaviors, and changing policy to make party hosts
responsible for incidents of underage drinking are all examples of community changes a

coalition can facilitate.

l6

Evaluation of Project Freedom, a substance abuse coalition in Wichita, KS
(Fawcett et al., 1997) describes how community change relates to achievement of long-
term population-level outcomes. The findings suggest that the coalition was engaged in
high levels of community mobilization and was effective at facilitating new or modified
programs, polices and practices targeted at their mission of preventing substance abuse.
Additionally, survey data from high school seniors showed a modest impact on reducing
use of alcohol. Examination of single nighttime vehicle crashes, a population level
indicator of community health, also suggests a negative correlation between this variable
and the number of community changes the coalition helped bring about. The researchers
investigated this relationship firrther, and discovered that rates of single nighttime vehicle
crashes were more likely to decrease when a sufficient number of community change had
been achieved, suggesting that the coalition contributed to reductions in crash rates.

Reexamination of the Fighting Back initiative with the perspective that coalitions
can facilitate necessary community changes that are important intermediate outcomes of
success sheds light on the contributions these coalitions made that were obscured by the
initial evaluation study by Hallfors and her colleagues (2002). Hingson et a1. (2005)
conducted a follow-up study of the Fighting Back Program, looking more carefully at a
subset of the communities that devoted a significant portion of their time and effort to
limiting access to alcohol and expanding treatment services. The researchers identified 5
of the 12 Fighting Back sites that brought about significant environmental and
community changes targeted at reducing alcohol availability and expanding treatment.
Examples of community changes included: conducting responsible beverage training;

enacting ordinances to prohibit public consumption or beverage sales; closing liquor

17

stores/blocking new stores/monitoring problematic outlets; limiting
marketing/advertising; increasing publicly funded treatment; initiating hospital
emergency room screens/referrals; establishing drug courts; and opening new
treatment/aftercare facilities.

These five communities, in comparison to matched controls and the remaining
Fighting Back communities, had more significant decreases in alcohol related fatal
crashes the 10 years following implementation of the Fighting Back program compared
to the 10 years prior, even taking into consideration the decreasing trend in fatal crashes
witnessed in recent years. The Hingson et a1. (2005) study indicates that effective
Fighting Back coalitions were able to bring about a higher number of community
changes, targeting both availability of alcohol and expanding treatment, and that these
community changes were precursors to reductions in alcohol-related crashes and
fatalities. What this study highlights is that coalitions vary in how effective they are, and
that when they are better community problem solvers, they bring about more community
changes. As such, greater focus needs to be placed on how to make coalitions better
community problem solvers. This study takes this next step, examining the role training
plays to increase a coalition’s capacity for community problem solving.

Community Problem Solving for Substance Abuse Prevention

One promising approach to help coalitions become effective change agents is the
community problem solving model used by Community Anti-Drug Coalitions of America
(CADCA) and its National Coalition Institute (Institute), a coalition training and capacity
building center that works with thousands of anti-drug coalition members across the

country. This model for community problem solving is based on work conducted by

18

coalition researchers at the Work Group for Community Health and Development at the
University of Kansas (KU Work Group) (Collie-Akers et al., 2007; Fawcett et al., 2001;
Roussos & F awcett, 2000). The KU Work Group created their model from the Institute
of Medicine’s framework for collaborative community health initiatives (Institute of
Medicine, 2002). According to this model, the aim of local coalition efforts is to modify
their community environments and systems, transforming targeted community sectors
such as the business, health and education sectors by changing programs, policies and
practices to increase the likelihood of healthier behaviors on a population-wide scale.
CADCA’s Institute has adapted this model for use with anti-drug coalitions on a nation-

wide scale. Its interpretation of the research-based model is depicted below in Figure 2.

l9

Figure 2. CADCA Institute’s Training Impact of Framework for Effective Community
Problem Solving

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Strong Coalitions Facilitation Achievement
Coalition Pursuing of Needed of
Internal Comprehensive Community Population -
Functioning Strategies Changes Level
Changes
0 Use of 0 Use of a L. 0 Changes in o Reductions
decision- —> variety of programs, —> in distal
making interventions policies behavioral
processes and actions and and health
0 Development focusing on practices outcomes
and use of individual 0 Changes in
planning level and more
products environmental sectors
0 Quality of change 0 Changes
coalition targeted at
membership more
0 Climate for groups
collaboration
A A it
CADCA CADCA CADCA
Training Training Training

 

 

 

 

 

 

 

 

 

In this framework, coalitions are effective vehicles for community health
promotion by taking a problem solving approach to addressing local needs and concerns.
Researchers emphasize that coalitions need to have a strong understanding of root causes
or local factors that explain the presence of substance abuse problems within the
community (Florin et al., 1993). This suggests that each community is unique, and as
such each community’s response to their substance abuse problem will be different. This

community problem solving approach is a process by which coalitions become local

20

experts on the unique local issues and conditions promoting substance abuse and
coalitions use local solutions to address the problem (Lasker & Weiss, 2003). As
depicted above, this model includes aspects of coalition inner work (i.e., internal
functioning), external work (i.e., implementation of comprehensive strategies), and role
as community change agents and their link as intermediate outcomes to population-level
change. While population-level outcomes are the long-term targets for coalitions, this
study focuses on how training improves capacity in the first three components: internal
coalition functioning, comprehensiveness of strategies, and facilitation of community
changes. Each of these is discussed below.
Coalition Internal Functioning

Coalition internal functioning refers to the inner work of the coalition, such as
activities and processes to create, improve and maintain the coalition (Butterfoss, 2007;
Foster-Fishman et al., 2001). Coalitions vary extensively in their internal coalition
functioning, and many studies have been conducted to understand the characteristics of
internal functioning found in effective coalitions (Roussos & F awcett, 2000; Watson-
Thompson, Fawcett & Shultz, 2008; Zakocs & Edwards, 2006). This research suggests a
well functioning coalition is more likely to achieve its goals. In the Community Problem
Solving Model, coalition internal functioning is comprised of four dimensions: 1)
coalition use of essential decision-making processes; 2) development and use of quality
planning products; 3) active membership involvement; 4) and a climate for collaboration.
The training approach targeted in this study is designed to help coalitions improve in each

of these four elements.

21

Coalition Use of Essential Decision—Making Processes

F awcett and colleagues propose detailed processes to help coalitions increase their
capacity to be effective community change catalysts (KU Work Group for Community
Health and Development, 2007; Watson-Thompson et al., 2008). Their review of the
coalition literature suggests “essential processes” that support community change and
improvement. Essential processes are “modifiable factors or activities that have been
shown to increase the likelihood of making an impact” (KU Work Group for Community
Health and Development, 2007,
http://communityhealth.ku.edu/ctb/explore_best_processes.shtml). By engaging in these
essential processes, coalitions are more likely to bring about changes in programs,
policies and practices specific to the community health problem they are tackling (KU
Work Group for Community Health and Development, 2007; Watson-Thompson et al.,
2008). The Institute’s community problem solving model trains coalitions in the
following evidence-based essential processes:

Assessing community needs and resources. This process refers to a coalition
engaging in a comprehensive assessment process to understand the needs and resources
in their community (Butterfoss, 2007). This is a fundamental component of most
coalition building models, such as the Framework for Participation for Community
Development (Habana-Hafner, Reed & Associates, 1999 as cited in Butterfoss, 2007) and
Community Coalition Model (Butterfoss, Goodman & Wandersman, 1993). Research
suggests that this is one important process coalitions need to invest in. For example,
evaluation of the Community Care Network indicated that the most successful

partnerships engaged in efforts to gather community input on the issue of health. These

22

communities sought to solicit information on the needs and concerns within the
community, some using focus groups, community forums and surveys (Shortell et al.,
2002). This process is important because it supports coalitions in implementing the right
strategies to address local concerns that fit community context (Foster-Fishman et al.,
2001; Shortell et al., 2002)

Analyzing information about the problem or goal. Researchers suggest that
coalitions examine and understand the nature of the problem affecting the community.
For example, problem analysis was one key process used by the community partnerships
in the W. K. Kellogg Foundation’s Turning Point Initiative to improve local public health
infrastructure (Lasker & Weiss, 2003). Some studies suggest that this may help a
coalition be more targeted in their approach by clearly articulating key problems to
address (Kreuter et al., 2000). This also sets the stage for selecting interventions based
on local context versus implementing strategies at random (Florin et al., 1993).

Developing a framework of change. In this process, the coalition specifies how it
will move from current conditions to improvements in population-level health and
behaviors. This process emphasizes the development of a visual map, such as a logic
model, to represent the coalition’s theory of action (i.e., the outcomes the coalition is
targeting and what they will do to achieve these outcomes) (KU Work Group for
Community Health and Development, 2007). Julian (1997) used a logic model in his
work with a United Way targeting changes in the human service delivery system. This
case study indicated that the logic model improved the organization’s planning and
evaluation processes by making explicit the community changes the organization was

targeting and the expected outcomes of such changes.

23

Developing and using strategic and action plans. This process helps the coalition
put more detail into its theory of action, specifying what will be accomplished, by when,
and by whom (Hays, Hays, DeVille, & Mulhall, 2000). The coalition selects specific
strategies to address identified problems. Kreuter and colleagues (2000) suggest that poor
planning that does not match strategies to outcomes is one reason coalitions never move
beyond implementation of intervention to the achievement of outcomes. Proper coalition
planning that delineates specific interventions to target local outcomes is one critical
process for a coalition.

Identifying, adapting and implementing interventions. Ofien called evidence-
based programs or best practices, coalitions need to implement a variety of interventions
that match their particular community setting (Foster-Fishman et al., 2001 ). The more
successful partnerships in the Community Care Network were ones that implemented
strategies that complemented each other in such a way that brought the coalition closer to
achieving the targeted aims of the community health initiative (Shortell et al., 2002). _

Evaluating the coalition. Achieving long-term outcomes requires time and
coalitions should track indicators to know if they are making progress toward their
mission and to improve their work (KU Work Group for Community Health and
Development, 2007 ; Roussos & F awcett; 2000). Most coalitions become involved in
evaluation because an external funder requires coalitions to track specified outcomes for
accountability purposes. However, when a coalition takes on an evaluation out of its own
curiosity about the effectiveness of their efforts and motivation to learn more and
improve, the evaluation process can be more successful (Wolff, 2003).

Sustaining projects and initiatives. A coalition needs to attune to the variety of

24

resources (funding, staff, material and other resources) needed to maintain and sustain
their coalition so that they are in existence long enough to bring about wide scale change
and improvements. Roussos and Fawcett’s (2000) review of the coalition literature
indicates that coalition success at securing financial resources for its efforts may predict
sustainability. For example, resources can be used to hire staff to help run and maintain
the coalition. While this may not be the most critical factor for coalition success (Wolff,
2001), a coalition’s ability to bring in human and financial resources may indicate some
level of strong functioning (Roussos & Fawcett, 2000).

The presence of these essential processes appears to be an indicator of effective
internal fimctioning. There is evidence to suggest that coalition use of these essential
processes promote community changes (KU Work Group for Community Health and
Development, 2007). Putting these essential processes to practice seems to be an
important function of coalition work and was measured in this study as one aspect of
internal coalition functioning.

Development and Use of Quality Planning Products

Another element of internal functioning is the development and use of quality
planning products. There is some support in the research literature for the development
of planning products to improve coalition functioning (F awcett et al., 2001; Hays et al.,
2000; Roussos & Fawcett, 2000; Watson-Thompson et al., 2008). In an empirical case
study of a coalition created to decrease risk for cardiovascular disease, one key
determinant of the coalition’s ability to facilitate these changes was the development of
planning products: a strategic plan that included vision and mission statements, a logic

model and an action plan (Collie-Akers et al., 2007). The distinction between the

25

Essential Processes and these Planning Products is that these are tangible documents to
guide the decision-making of the coalition. For example, while a coalition may decide to
implement a variety of interventions, mapping them on paper and having a “reference
guide” appears to be an important aid to coalition work. The Institute’s training approach
incorporates the development and use of planning products (Community Assessment,
Logic Model, Strategic/Action Plan, Evaluation Plan and Sustainability Plan) to help
coalitions engage in the essential processes in their community. This study incorporated
a measure of coalition use and development of quality planning products as a measure of
internal functioning critical to a coalition’s community problem solving capacity.
Active Member Involvement

The importance of coalition member involvement is highlighted in a variety of
coalition fiameworks, such as the Conceptual Framework of Coalition Assessment
(Sofaer, 1993 as cited in Butterfoss, 2007), Framework for Partnerships for Community
Development (Habana-Hafner, Reed and Associates, 1989 as cited in Butterfoss, 2007)
and the Community Coalition Action Theory (Butterfoss & Kegler, 2002 as cited in
Butterfoss, 2007). Membership is important because “members are the lifeblood of a
coalition-without them, there is no organization at all. . .members determine the vision,
the course and the outcomes of effective coalitions” (pg. 138, Butterfoss, 2007).
Coalitions by definition bring different stakeholders together to work in collaboration
toward a mutually agreed upon goal. These stakeholders or members then do the work of
the coalition, each within their realm of influence, contributing varied skills,
relationships, knowledge and resources. As such, coalitions with greater diversity in their

membership may also have more expertise and assets to put into play at addressing the

26

targeted community concern (Foster-Fishman et al., 2001; Mays, Halverson & Kaluzny,
1998 as cited in Lasker, Weiss & Miller, 2001).

Research has put attention into examining the role membership plays in coalition
functioning, examining both the diversity of membership involvement (i.e., how many
different groups/individuals are represented) and how actively they are involved (Allen,
2005; Allen, 2007; Hays et al., 2000). While diversity of membership is important, the
degree to which these stakeholders are actively involved may be the more critical factor
related to coalition member involvement. For example, greater member participation may
require greater coordination of efforts in order to keep members engaged, so a key issue
may be how successfully a coalition can engage members in the work of the coalition.
Allen’s (2005 and 2006) study of coalitions addressing domestic violence supports this
premise. Her study found that active member participation was a predictor of council
effectiveness (i.e., making progress on intermediate and long-term goals). This study
measured breadth of active participation as a key component of coalition internal
functioning.

Coalition Climate for Collaboration

Much attention has been given to the relationship between the quality and degree
of collaboration within a coalition and coalition functioning (Foster-Fishman et al., 2001;
Lasker, Weiss & Miller, 2001; Weiss, Anderson, & Lasker, 2002). While coalitions
bring diverse stakeholders, skills, viewpoints and resources to the table, coalitions also
need to create a climate that supports and promotes these diverse members in working
together toward a common goal (Lasker et al., 2001).

High functioning coalitions create a climate for collaboration (Foster-Fishman et

27

al., 2001; Zakocs & Edwards, 2006). Foster-Fishman and colleagues (2001)
conceptualized this as positive internal relationships illustrated by characteristics such as
trust, cohesiveness, common vision, collective decision-making, and effective conflict
resolution processes. When this internal climate exists, members may feel more
comfortable taking risks together, sharing different viewpoints, transforming conflict
fi'om fighting to growth and learning, and participating more firlly in the coalition effort
(Butterfoss, 2007; Chavis, 2001; Lasker & Weiss, 2003). The ability of coalitions to
create such a supportive climate for their work appears to be a critical element of
coalition internal functioning and was included in this study as one of the four
components of a well functioning coalition.
Comprehensiveness of Strategies

Another key factor that characterizes capacity for community problem solving is a
coalition’s ability to engage in multi-strategy, comprehensive action. The convergence of
community stakeholders is not an end in and of itself. The real work of an effective
coalition is its efforts to change community norms, attitudes, behaviors, systems and
environments. A single intervention is unlikely to achieve the high level of community
transformation needed to improve health and well-being (Merzel & D’Affitti, 2002).
Instead, ecological theory proposes that behavior, such as substance use and abuse, is
influenced at multiple levels and as such interventions must be broad-based,
comprehensive and seek change at multiple levels (Bronfenbrenner, 1979; Sorensen,
Emmons, Hunt & Johnston, 1998). In addition to changing individual-level factors,
coalitions need to engage in strategies to improve conditions in families, organizations,

systems and communities. Coalitions need to take action using a wide array of

28

interventions at its disposal, including advocacy efforts and implementation of evidence-
based interventions (F awcett, Francisco & Schultz, 2004).

Comprehensiveness of strategies, in this study, refers to actions the coalition
engages in to address substance abuse in the community (i.e., strategy implementation).
It involves all the planning, meeting, coordinating, mobilizing, etc. that a coalition
engages in to bring about a community change. This may be working on changing a
local policy or planning with schools to implement a Student Assistance Program, a
school-based program to prevent, identify, and intervene in areas of identified needs that
may have a negative impact on academic achievement and youth development, including
substance abuse. These are the actions coalitions take in their work toward bringing
about a community change

Examination of substance abuse coalition actions suggest that the primary areas of
coalition intervention are in information dissemination, such as education about the
harms associated with substance use, building individual skills, such as communication
and drug refusal competencies, and providing support, such as drug-free alternative
activities (Florin et al., 1993; Hallfors et al., 2002). Less emphasis appears to be in
interventions targeting systems change, such as increasing enforcement of underage
drinking laws, prohibiting sales of drug paraphernalia, or advocating for responsible
beverage server trainings in bars and restaurants. Florin and colleagues encouraged
training and technical assistance providers to build coalition capacity to employ a variety
of strategies, not just those focusing on individually-focused interventions, such as
information dissemination.

Some effort has been made to create a typology for coalition prevention activities,

29

such as the one used by Florin and colleagues (1993) that categorized coalition activities
into eight areas, including drug flee alternative activities, information dissemination,
referral and treatment, and reducing environmental support for drugs. Roussos &
Fawcett (2000) suggest that change strategies can be categorized by intensity, suggesting
that information dissemination activities are “weaker” than those that focus on changing
access to health care or implementing new policies. These activities are weaker in the
sense that they focus more on individual-level change instead of community and systems-
level changes. They and their colleagues have put effort into categorizing behavior
change strategies (i.e., coalition interventions) into the following areas, ranging from less
intense interventions to more intense activities that are considered to be “environmental
strategies” in the substance abuse prevention world (Paine-Andrews et al., 2002):

0 Providing information - Educational presentations, workshops or seminars, and
data or media presentations (e.g., public service announcements, brochures,
billboard campaigns, community meetings, town halls, forums, Web-based
communication).

0 Enhancing skills - Workshops, seminars or activities designed to increase the
skills of community members, such as youth, parents and citizens (e. g., training,
parenting classes, evidence-based prevention programs for youth).

0 Providing support - Creating opportunities that reduce risk or enhance protection
(e. g., providing alternative activities, mentoring, support groups, youth clubs,
parenting groups, Alcoholics or Narcotics Anonymous).

0 Enhancing access/reduce barriers - Improving community and service delivery

processes to increase the ease, ability and opportunity for community members to

30

access and use the services (e.g., access to treatment, childcare, transportation,
housing, education, special needs, cultural and language sensitivity).
Changing consequences — Using incentives and disincentives to alter
consequences of a specific behavior (e. g., increasing public recognition for
deserved behavior, individual and business rewards, taxes, citations, fines,
revocations/loss of privileges).

Changing the physical design of the environment - Altering the physical structure
of the community so that individuals are less likely to engage in substance use
(e.g., parks, landscapes, signage, lighting, outlet density).
Modifying/changing policies - Working to create formal changes in written
procedures, by-laws, proclamations, rules or laws with written documentation
and/or voting procedures (e.g., public policy actions, systems change within
government, communities and organizations).

As part of this study, coalition internal functioning included a measure of the

degree to which coalitions engaged in a multitude of change strategies, with the view that

greater comprehensive indicated a coalition with greater capacity for community problem

solving. This built upon the existing viewpoint that coalitions need to expand beyond

purely information dissemination and providing support to other strategies that focus

more on systems and environmental change. This study assessed the degree to which

coalitions employed different types of community-based strategies and interventions.

Facilitation of Community Changes

The final component of the problem solving model examined in this study is the

extent to which coalitions facilitated community changes, defined as new or modified

31

programs, policies and practices that are related to the mission and goals of the coalition
(Francisco, Paine & F awcett, 1993; Fawcett et al., 1995). These differ from the change
strategies or actions described in the above section. Here, community changes are the
result of the coalition actions and strategy efforts. While a coalition may work to
mobilize a community to pass an ordinance, it is not considered a community change
until the ordinance is officially passed. The work prior to the passing of the ordinance is
considered to be strategy implementation (Francisco, Paine & F awcett, 1993; F awcett et
aL,1995)

Examples of community changes include translation of an evidence-based
prevention program to Spanish to meet the needs of a growing Hispanic population
(programmatic change); working with physicians to include discussions of substance
abuse with parents and youth during school check-ups (practice change); and
development and implementation of local ordinance that bans the sale of alcohol during
community festivals (policy change). While this may not be the distal outcome for a
coalition, community changes facilitated by coalitions are an important intermediate
outcome to long-term outcomes (Allen et al., 2008; Hays et al., 2000; Roussos &

F awcett, 2000)

Collie-Akers and colleagues (2007) used an empirical case study design to
determine if the Kansas City-Chronic Disease coalition was able to help bring about
community changes to decrease risk for cardiovascular disease and diabetes among
Hispanics and Afiican Americans. Community changes were documented and analyzed
to determine if the coalition was successful at changing community systems and

environments. Examples of community changes this coalition helped facilitate include a

32

 

new physical activity program and implementation of a smoke-free workplace ordinance.
The researchers discovered that the coalition helped facilitate 321 community changes
within a two year period of time, suggesting coalitions can be very actively engaged in
facilitating community change.

However, while facilitating change is important, not all community changes lead
to population-level outcomes. Hingson et al.’s (2005) follow-up study of the Fighting
Back Initiative indicated that all 12 sites were able to facilitate some community changes.
However, what differentiated the five Fighting Back sites with significant reductions in
alcohol related crash fatalities was that they facilitated more community changes (at least
8) and they targeted both reducing alcohol availability and expanding treatment. The
other seven sites facilitated fewer changes that mostly targeted treatment expansion. It is
critical, then, to think about the scale and scope of the community changes that are
brought about.

The research conducted by the KU Work Group provides some suggestions on
how community change may trigger population-level outcomes (Paine-Andrews et al.,
2002; Roussos & F awcett, 2000; Watson-Thompson et al., 2008). The KU Workgroup
has developed a monitoring system so coalitions can document the community changes
brought about through their collaborative efforts (F awcett et al., 2001; Fawcett et al.,
1995). Their work with coalitions suggests that population-level changes are more likely
when community changes are greater in amount (i.e., number of changes), intensity (i.e.,
use of more behavior change strategies that go beyond information dissemination and
skills building), duration (i.e., changes are more long-term instead of “one-shot” events),

and exposure (i.e., penetration across the whole community and targeted populations)

33

(Roussos & Fawcett, 2000). For example, Paine-Andrews and her colleagues (2002)
measured the community changes that two Midwest neighborhoods helped bring about to
prevent teenage pregnancy over the course of four years. They found that reductions in
teen birth rates were greater in the community that produced more community changes.
These changes were also more comprehensive (targeting more risk factors and program
components) and distributed among more sectors (e.g., business, health and education)
and target groups (e.g., youth, families, community leaders).

In summary, community change appears to be a viable measure of coalition
intermediate outcomes and offers a more detailed insight into the process by which
coalitions contribute to long-term population-level improvements in community health
and well—being (Roussos & Fawcett, 2000). This study examined coalition facilitation of
new or modified programs, practices and policies for substance abuse prevention. This
study was not able to track all elements of community change used in the KU Model.
However, this study did allow for the assessment of community engagement as change
agents in a large study. As such, examination of the targets and sectors of change was
included, with the expectation that coalitions with stronger community change capacity
promoted change in more areas and with more targets. In addition, this study assessed
the degree to which the community changes addressed the mission, goals and community
context of the coalition.

In conclusion, this study included three key elements that define a coalition’s
capacity for community problem solving: coalition internal functioning, use of
comprehensive strategies, and promotion of community changes. By engaging in this

community problem solving model, coalitions work through aspects of internal and

34

external functioning that help promote their ability to be community change agents,
actively engaged in facilitating new or modified programs, policies and practices. High
levels of capacity in each of these elements of community problem solving increase
coalition potential to reach the final step of the model — contribution of their efforts to
population-level change in substance abuse outcomes.

The Study Context

CADCA Institute training is designed to improve coalition capacity in each
component of the community problem solving model described above. This large
national training center provides the context for this study, and before a discussion of
how well designed training can improve coalition problem solving capacity is provided, a
description of the study setting is useful so that the discussion is framed within a context.

CADCA is a national organization that works to increase the capacity of
coalitions to comprehensively address their local substance abuse problems. CADCA’s
National Coalition Institute (Institute) was created to support the development and
sustainability of local anti-drug coalitions. The Institute, incepted in October of 2002,
has developed a comprehensive training and technical assistance system for substance
abuse coalitions using the community problem solving model described in the previous
section.

Capacity for community problem solving, as defined by CADCA, includes strong
internal functioning, use of comprehensive strategies and facilitation of community
changes. CADCA trains continuously on each component of its community problem
solving model using trainings of various lengths and topics. A description of the

Institute’s training content for community problem solving is provided in Table 1.

35

Table 1. Description of CADCA Institute Trainings to Build Coalition Capacity for
Community Problem Solving

 

COALITION INTERNAL FUNCTIONING

 

Essential Processes

0 Training on each of the 7 essential processes (Assessing community needs and resources, Analyzing
information about the problem or goal, Developing a framework of change, Developing and using
strategic and action plans, Identifying, adapting and implementing interventions, Evaluating the
coalition, Sustaining projects and initiatives)

0 Providing action steps for each process (e.g., steps to conducting a comprehensive community
assessment)

0 Demonstrating how each process builds upon each other

 

Planning Products

0 Providing templates for each product with core elements associated with each

0 Teaching coalitions how to develop each product and use it when they return to their communities.

0 Discussing how to use these products to guide the decision-making processes of the coalition.

o Emphasizing that products need to be revisited periodically so that they can be updated to
continually be a resource to the coalition

 

Membership Involvement

o Diffusing the work of the coalition across all members

0 Creating a plan that designates how all the coalitions members will contribute to the strategic efforts
of the coalition to comprehensively address substance abuse

 

Collaborative Capacity

0 Developing consensus and shared understanding of the drug abuse problem
0 Creating and maintaining coalitions and partnerships

0 Improving coalition operations

 

COMPREHENSIVE STRATEGIES

 

0 Understanding how to select and implement evidence-based, comprehensive interventions
0 Matching interventions to data on local context
0 Emphasizing use of environmental strategies that target systems changes

 

ACCOMPLISHING COMMUNITY/SYSTEMS CHANGES

 

0 Defining community change (i.e., new or modified programs, policies or practices)
0 Tracking community change
0 Matching community change to coalition mission and aim

 

All components of the Institute’s community problem solving model are
imbedded into its trainings, as seen above in Table 1. The training is also highly
experiential, usually providing information in large groups and then giving coalitions
time in small group sessions to practice the information covered. The Institute strongly

encourages the training participants to return to their communities and teach other

36

coalition members the lessons learned in the training. Transfer of training, that is use of
training information back within the real-life context, is very difficult to achieve
(Baldwin & Ford, 1988; Rhodes, Lok, Hung, & Fang, 2008). The intent of Institute
training is not only for individuals to use lessons learned back in their communities but to
also disseminate the information to the rest of their coalition. In order to make sure
learning goes beyond the individual, the Institute helps foster coalition-wide use of
training information by providing information on how to encourage transfer. The
planning products are the key ways this occurs by providing participants opportunities to
learn how to develop the products and then take them back home to facilitate the product
development process with their other coalition members and staff. The Institute’s
external evaluation suggests that training participants do share information with coalition
members, indicating some level of transfer of training is occurring (Foster-Fishman,
Nowell, Archer & Greenway, 2005).

Summaries of yearly Institute training figures are provided in Table 2. The
Institute only considers an event as training if it lasts one day or more in length.

Table 2. Summary of Institute Trainings

 

 

 

 

 

Fiscal Year # of Institute trainings # of coalitions / organizations
conducted trained
October 2004 — September 2005 16 538
October 2005 — September 2006 27 748
October 2006 — September 2007 41 887
October 2007 — September 2008 51 1079

 

The Institute offers coalition trainings that vary in length, topic and format level.
Most trainings are two to four days in length, covering the information described in Table
1. Examples of Institute trainings are described below in Table 3. The Institute provides

training in different venues — community, state, regional and national level trainings.

37

Almost all Institute trainings are free to coalitions. While coalitions need to pay travel
expenses to attend the training, the majority of trainings do not have a registration fee.
Some trainings, such as the National Coalition Academy (please see Table 3 for
description) cover housing and food costs for the duration of the training.

Table 3. Examples of CADCA Trainings

 

Length of

Training Name Description Training

 

Basic level training intended mostly for new coalitions.

 

C l'ti . . . . .
oa l .0“ Core Provrdes foundational information on community 1-4 Days
Essentials .
problem solvrng.
Advanced level training that builds upon the core
Advanced Issues in essentials class, delving more deeply into how coalitions
Coalition Building —~ can track community changes and analyze their 1-2 Days
Evaluation contribution to population-level substance abuse

reductions.

 

Advanced level training that builds upon the core

Advanced Issues in . . . . . .
essentials class, provrdmg more m~depth rnforrnatron on

 

 

 

Coalition Building - . . . . . 1-2 Days
. . . how coalitions can proactively sustain their human,
Sustamabrlrty . .
financral and programmatic resources.
. ‘ ' 4 1
National Youth Core essentials class adapted fora youth ( 8 years and
. . . . younger) audience. Youth are trained alongsrde an adult
Leadership Imtratrve . 4 days
counterpart on all aspects of community problem
(NYLI) .
solvrng.
Advanced level training in which coalitions provide a
formal presentation on how they have infused
From Theory to . . . . . .
. community problem solvrng into their coalition work. 1-2 Days
Practice . . . . . . .
Coalitions are critiqued on the quality of coalition work
and clarity in “making a case” for their coalition.
Intensive year-long training experience that blends 3
weeks of face-to-face training on community problem
National Coalition solving, distance learning to promote training transfer 1 car
Academy (NCA) between in-person training sessions, and use of an online y

documentation system to track the community changes
the coalitions facilitates.

 

 

 

38

Most Institute trainings offered cover most of the core elements of the framework
for effective community problem solving depicted in Figure 2. Additionally, even though
various trainings with various lengths, levels and topics are offered, the trainings all link
back to the core elements listed in Table l. The Institute is very consistent in ensuring its
trainings link back to the basic foundation of its community problem solving framework.
For example, an advanced training may provide more in-depth information on a topic, but
it is grounded in the core elements of the Institute’s basic community problem solving
model. In this way, every Institute training can be considered a booster training, each
supporting the previous training. While training may be designed in a developmental
sequence, regardless of the stage, the core elements form the foundation for all Institute
trainings.

The Institute has invested time and effort to evaluate the effectiveness of its
training approach. Initial evaluations had a formative focus, assessing the degree to
which the Institute was developing its capacity to provide training on a sufficient scale
and scope and with sufficient quality to help coalitions at the nation-wide level (Foster-
Fishman, White & Droege, 2007, Foster-Fishman et al., 2005). These qualitative
evaluations assessed training participant satisfaction with Institute trainings and
development and use of core planning products to inform decision making. The
evaluations suggest that coalitions trained by the Institute improve their functioning. The
most recent qualitative evaluation occurred for the time period running October 1, 2005-
September 30, 2006 (F oster-F ishman et al., 2007). Of the 32 coalitions interviewed, 89%
indicated that the training helped them think more comprehensively and strategically,

improving decision making processes. A more recent quantitative evaluation of the

39

Institute’s training and technical assistance supports the Institute’s community problem
solving model (Foster-Fishman et al., 2008). A structural equation model conducted with
391 coalitions that participated in CADCA’s 2007 Annual Survey of Coalitions (a
coalition survey administered by CADCA) indicated that the theory of change depicted in
Figure 1 has good model fit. Institute training and technical assistance appear to trigger a
process by which coalitions improve their coalition internal functioning, implement
comprehensive strategies and facilitate community changes. Additional analyses also
indicate that the coalitions that receive training and technical assistance from CADCA
develop or revise more of the planning products and also report using them more
frequently to inform their decision-making processes.

These findings are promising and provide some evidence to suggest the Institute
is an effective training and support organization for substance abuse coalitions.
However, while it appears that CADCA training can improve coalition community
problem solving capacity, neither the Institute evaluations to date nor the existing
coalition training literature provide sufficient information on how coalition outcomes
may vary as a result of different amounts of training participation. Training is resource
intensive in terms of time and money. Coalitions need to know how much investment
they should put into training in order to get the maximum benefit. Understanding what
aspects of training participation are most important to maximize the benefit of training is
a question left unanswered by current research on training and was the pivotal focus of
this study.

Relationship between Training and Coalition Community Problem Solving

The previous section points to a variety of areas in which coalitions need to build

40

their competency. There is substantial evidence to suggest that coalitions require an
extensive amount of skills and knowledge in order to be effective. For example,
coalitions need to foster collaboration among their stakeholders, track their outcomes,
develop and implement a strategic plan, and build both internal and external capacity
(Butterfoss, 2007; Foster-Fishman et al., 2001; Roussos & Fawcett, 2000; Watson-
Thompson et al., 2008; Zakocs & Edwards, 2006; Zakocs & Guckenburg; 2007).

Training and technical assistance are common ways to help build this coalition
capacity, and research supports the utility of training for coalitions (Butterfoss, 2004;
Butterfoss et al., 2003; Florin et al., 1993; Kegler et al., 1998). Additionally, coalition
training is often mandated by funders to build knowledge and skills so that coalition
grantees can achieve the goals of the grant program (Department of Health and Human
Services RFA No. SP-09-002; Florin et al., 1993). Other research suggests that capable
coalitions understand how they can benefit from this outside support and proactively seek
it out when needed (Russell, 2002 as cited in Butterfoss, 2007; Roussos & Fawcett,
2000). Training research outside of the coalition field suggests that overall, investment in
training has benefits for employees and their organizations. Reports on learning
investments in Fortune 500 companies by the American Society for Training &
Development suggest that more successful companies invest more money and time in
training their employees (Paradise, 2007; Rivera & Paradise, 2006). Additionally, there
is evidence that entry-level training is associated with better job attitudes, coping ability,
job performance and fewer intentions to leave the organization (Saks, 1996).

The need for coalitions to utilize training and technical assistance to develop and

maintain critical skills draws parallels to the culture created in learning organizations.

41

The studies performed in the learning organization field emphasize the adoption of a
culture that values learning and growth. Learning organizations institutionalize a deep
cycle of learning that is critical for sustainable change (Senge, 1990). One part of
learning is being able to do new things. This comes about through acquisition of skills
and capabilities. Applying learning organization theory to coalitions suggests that
coalitions can benefit from training and technical assistance that help them learn and
build the skills, tools and knowledge needed to thrive and succeed in their communities
(Butterfoss, 2007).

Research on coalition training indicates a beneficial effect for coalitions. Training
can help coalitions more actively engage in advocacy efforts and improve communication
and collaboration (Stevens and Lodl, 1999). Coalition training can not only build
knowledge such as greater understanding of prevention principles but also improve
coalition readiness and internal and external coalition fimctioning (Feinberg etal., 2002).
Since not all coalitions are effective at building their skills and capacities and meeting
their goals, as depicted in the Fighting Back studies (Hallfors et al., 2002; Hingson et al.,
2005), training appears to be an essential ingredient to help coalitions build community
problem solving capacity. This study continued existing inquiry on the effectiveness of
training in building skills and competencies critical for coalition success. This was done
by examining if Institute training improved a coalition’s capacity for community problem
solving. It was hypothesized that those coalitions trained by the Institute would have
greater capacity for community problem solving than those not trained by the Institute.

HYPOTHESIS: Coalitions that have received training from the Institute will be

stronger in each element of community problem solving capacity than coalitions not

42

trained by the Institute.
Dose-Response Theory

While current coalition training research indicates that training is helpful, if not
necessary for coalition outcomes, very little research has been conducted on how much
training is needed to benefit the coalition, how training amount should be defined or how
to model the relationship between training amount and coalition outcomes. The purpose
of this study was to not only determine if coalitions improve as a result of training but
also to describe the association between training quantity and coalition capacity for
community problem solving. Specifically, this research proposed to examine the
relationship between training amount or dosage and coalition capacity for community
problem solving as measured by increased coalition internal functioning, use of
comprehensive strategies, and facilitation of community changes.

To investigate this relationship, this study was the first to apply dose-response
theory to the coalition training field. Dose-response theory describes “the relationship
between amount of exposure to the intervention and amount of improvement in observed
outcomes” (Valentine, Gottlieb, Keel, Griffith & Ruthazer, 1998, pg. 366). While other
disciplines such as education (Chaput, Little & Weiss, 2004) and program
implementation (Durlak & DuPre, 2008) have applied this concept to their work, dose-
response theory has not been applied to training, particularly in the coalition training
arena, except in the most limited sense. Understanding how training dosage relates to
outcomes has both empirical and practice value. Empirically, common knowledge is
increased on how to design training to support coalition capacity to achieve long-term,

population-level reductions in substance abuse. The applied sector would benefit from

43

this research as well, using research to inform the wise use of scarce training dollars.

There is a growing body of evidence across multiple disciplines to suggest that
higher amounts of an intervention received are related to better outcomes. For example,
the psychotherapy literature suggests that mental health consumers improve with more
treatment sessions (Hansen, Lambert, & Forman, 2002; Howard, Kopta, Krause, &
Orlinsky, 1986). Prevention scientists working to reduce adolescent aggression and
youth substance abuse have examined the relationship between program dosage (as
measured by program attendance) and program outcomes, with overall findings
suggesting a link between more exposure to the prevention program and better outcomes
such as less absenteeism, decreases in school failure and reductions in aggressive
behavior (August, Lee, Bloomquist, Realmuto & Hektner, 2004; Charlebois, Brengden,
Vitaro, Norrnandeau, & Boudreau, 2004; Valentine et al., 1998).

Some researchers have also hypothesized that insufficient dose is one reason why
interventions have not been shown to be effective (August et al., 2004; Charlebois et al.,
2004; Rescinow et al., 2001). In intervention research, the dosage question focuses on
how much of an intervention an individual received. Ignoring the role intervention
dosage plays in impacting outcomes increases the likelihood of Type III error —
incorrectly ruling an intervention as ineffective when the intervention was not
implemented with sufficient fidelity, intensity or consistency (Dane & Schneider, 1998;
Rescinow et al., 2001). Low dose may also explain why intervention outcomes are not
sustained over time. Program participants may receive a sufficient dose to have some
benefit but not a high enough of a dose to maintain these benefits over a period of time

(August et al., 2004). Across fields, researchers are calling for more studies examining

44

the relationship between dose and outcomes in order to achieve and improve benefits for
targeted populations (Bickman, Andrade & Lambert, 2002; Chaput et al., 2004; Nation et
al., 2003). Due to the serious consequences of ignoring exposure to the intervention,
researchers are now emphasizing the importance of implementing a careful process to
measure participant dose (Chaput et al., 2004; Rescinow et al., 2001).

Framing coalition training using a dosage model allows the expansion of current
understanding of the usefulness and appropriateness of training for coalitions. This has
both scientific and practical implications. For coalition research, expansion of our
understanding of the conditions under which training is most beneficial expands our
knowledge of how to build coalition capacity for community problem solving to
ultimately achieve population-level improvements in community health and well-being.
Additionally, understanding how different types of training dosage may lead to different
coalition outcomes improves coalition building and increases existing knowledge of
effective training design (e.g., length and intensity of training). The availability of well-
dosed training with specific targeted outcomes for coalitions is essential for community
change and promotion of health and well-being. In applied settings, in addition to
evidence on the effectiveness of a particular training, more precise information on the
optimal dosage of training needed to achieve outcomes improves coalition decision-
making, especially around cost planning. Trainings come in various lengths, topics and
frequencies. Additionally, face-to-face training is expensive and time intensive. For
example, one well known coalition training model, the Communities That Care (CTC)
system, lasts approximately eight days, which may be divided into multiple sessions,

requiring either travel to a designated training site or paying for trainers to train onsite in

45

the community. When CTC training was offered by a for profit entity, the coalition
training package cost at least $25,000 not including travel and other expenses (Channing
Bete, 2001). When most coalition budgets are less than $200,000 (Foster-Fishman et al.,
2008), this is a substantial, even cost-prohibitive, investment in training. Even when
training is free, there may be associated travel costs, including airfare, mileage, lodging
and meals. Indirect costs also exist when time away from work is calculated. Improper
investments of valuable time and resources may be made in training if coalitions send too
few or too many individuals to a training or attend an insufficient number of trainings to
benefit the coalition. This study, for the first time, empirically examined types of dosage
that are relevant to coalition training and how these dosage types relate to improvements
in the three elements of community problem solving of interest.
Defining Training Dosage

Since very little has been done in training specifically to understand and measure
dosage, knowledge from other fields must be drawn upon to inform the understanding of
how to conceptualize training dosage for coalitions. A review of the literature across
disciplines, including psychotherapy, out-of-school time programs (OST), prevention and
intervention programs, media campaigns and toxicology, offers a variety of ways to
measure dose. Although many focus on the amount of an intervention received (Chaput et
al., 2004; F iester, Simpkins, & Bouffard, 2005; Valentine et al., 1998), some researchers
suggest dose has two aspects — quantity and quality (Nation et al., 2003; Rescinow et al.,
2001). While quantity measures some aspect of amount of exposure, quality tries to
capture some aspect of the integrity or fidelity of the intervention received. Applied to

coalition training, training quantity might measure how much time a coalition spends in

46

training and quality might measure how well the training was implemented to training
guidelines. Future inquiries may investigate training dose quality; however as a first step
in examining the dose-response relationship for coalition training, this study examined
the quantity of training coalitions need to increase their capacity for community problem
solving

The dose-response relationship for coalition training has received only a limited
amount of attention from coalition researchers. CTC researchers have made an initial
attempt to investigate the relationship between attendance at CTC trainings and project
outcomes. The CTC model trains communities to bring key leaders together, build their
understanding of prevention science and the risk and protective factors associated with
youth problems, implement a survey to assess each community for its specific risk and
protective factor profile, and design a plan targeted at implementing evidence-based
programs to address the community’s identified risk factors (F einberg et al., 2002;
Quinby et al., 2008). CTC training dosage has been assessed at both the individual level
(i.e., number of trainings an individual attended) and community level (i.e., the number
of participants per community per training).

Results at the individual level show very marginal relationships between
individual—level training attendance and individual-level outcomes, indicating only an
association with greater individual-level understanding of the CTC process (F einberg et
al., 2002). On the other hand, community-level analyses performed to assess if the
number of training participants per community are related to community-level outcomes
show greater linkages between training attendance and outcomes (Feinberg et al., 2002;

Quinby et al., 2008). For example, higher levels of attendance are related to better

47

 

coalition capacity outcomes, such as better understanding of the CTC system, community
readiness, higher attendance at board meetings and more time involved in the CTC
process (Quinby et al., 2008).

These findings imply that training is linked positively to both internal and external
coalition functioning (F einberg et al., 2002; Quinby et al., 2008). Additionally, the
researchers suggest that coalition training effects operate in a group-level manner and
when more members are trained, this training information becomes institutionalized in
the community. While individuals are the entities being trained, the CTC studies suggest
that there is value in aggregating this information to create a coalition-level training
dosage score. The current study continued this line of coalition training inquiry, using
coalition-level training dosage measures and examining how coalition outcomes varied
with coalition-level exposure to training. It should be noted that this study takes an
individual-level concept of training dosage and expands its use to the organizational (i.e.,
coalition) level. Implications of this approach will be discussed in more detail in the
subsequent section.

The CTC studies offer an initial start at conceptualizing training dosage and its
relationship to coalition community problem solving capacity. However, since research
in the area is sparse, other fields must be examined to understand in what additional ways
training dosage can be defined. Much of the work in conceptualizing dose quantity has
been done in areas that focus on individual-level change, such as out-of-school time
(OST) programs, prevention programming, psychotherapy and media/marketing for
health promotion (Chaput et al., 2004; Howard et al., 1986; Wray, Jupka, Ludwig-Bell,

2005). As the CTC study (Feinberg et al., 2002) indicates training attendance at the

48

coalition-level appears to have more relevance to achievement of coalition outcomes than
individual-level attendance. Translating the existing research on training dosage from
more individual-level measures to organizational or coalition-level measures is needed.
This study took the first steps at applying dose measures to coalition training in a
comprehensive manner.

When determining how coalition training dosage or exposure should be
operationalized, some guidance is provided in existing research outside of training. Dose
or exposure in the fields of OST programs, psychotherapy, intervention and prevention
research and marketing have been measured in a variety of ways, including intensity of
exposure (e.g., viewing health campaign ads twice a week), duration of exposure (e.g.,
involvement in an after-school program for three years), fiequency of exposure (e. g.,
receipt of 16 therapy sessions), and breadth of exposure (e.g., participation in 4 out of 6
of the program activities) (Chaput et al., 2004; Howard et al., 1986). While helpful to
know that dosage can be operationalized in a variety of ways, the key issue for this
research study was to hone in on relevant key indicators of dosage for coalition training.
For the purposes of this study, a comprehensive examination of training dose included the
following indicators:

0 Breadth of exposure — total number of unique individuals trained per

coalition
0 Frequency of exposure — total number of individuals attending more

than one training event

49

 

o Intensity of exposure — average number of individuals attending

coalition trainings (i.e., sum of training attendance across all trainings
divided by total number of training events attended)

In choosing the above three measures of training dose for this study, careful
consideration of existing literature and the context for the study played a role. This study
attempted to create a theory for coalition training dosage where there is none, and as such
drew upon the lessons learned from other fields as was appropriate. Additionally, since
this study occurred within the real-life context of a training Institute, the measures of
dosage had to be contextually relevant and useful. Institute staff considered these three
types of training dosage relevant to their work and desired more information on how
variations in these dosage measures may produce different coalition outcomes. The three
indicators of training breadth, frequency and intensity balance both science and practice
to create a more meaningful and useful application of dose-response theory to the area of
coalition building. These three types of dosage were potentially confounded by factors
such as coalition size and budget, and these variables were included as covariates in this
study. The three dosage types examined in this study are described below.

Breadth of Exposure

One measure of dosage relevant for coalition training is breadth of exposure
defined as the number of individuals from the coalition that have ever attended a training.
This measure of training attendee participation is appropriate based on the Communities
That Care (CTC) study findings (Feinberg et al., 2002; Quinby et al., 2008) that indicated
that the number of individuals attending a training from each community was positively

related to CTC coalition outcomes. Feinberg and colleagues summed the number of

50

individuals attending each of the three CTC trainings and used the total number of
attendees per community as the independent variable to predict overall community
understanding of prevention. The researchers also conducted analyses to understand how
community-level attendance at each training predicted different training outcomes, such
as community readiness, CTC efficacy and internal and external functioning. For
example, greater community-level attendance at the Key Leader Training is associated
with more perceived community readiness, internal and external functioning, belief that
prevention works, and belief that the CTC program will sustain itself in the future.
Higher attendance at the Risk/Resource Analysis training is positively related to
community readiness and external influence but negatively related to sense of purpose
and perceived efficacy of the CTC leadership. For the Promising Approaches Training,
community-level attendance is associated with less community risk and a stronger
understanding of conditions that are not contributors of youth behavior problems
(Feinberg et al., 2002).

A measure of breadth of exposure to training attendance was used in this study.
This was a count of the total number of unique individuals from each coalition trained by
the Institute. Unlike CTC trainings which are modular in format (i.e., three different
trainings covering different topics that comprise the entire CTC curriculum), most
Institute trainings cover all the relevant information critical to community problem
solving in one training event. As such, an overall measurement of breadth of exposure
across all trainings and its relationship to elements of community problem solving
capacity was examined. It should be noted that this type of coalition dosage may be

confounded by coalition size since larger coalitions have more people they can send to

51

 

trainings. As such, size was included in this study as a covariate.
Frequency of Exposure

Measures of fiequency of participation have been found to be useful in other
studies, such as those that have tracked exposure by counting the total number of sessions
or days (or any other appropriate unit) received by intervention recipients (August et al.,
2004; Liaw, Meisels, & Brooks-Gunn, I995; Nation et al., 2003; Valentine et al., 1998).
The Infant Health and Development Program (IHDP), a randomized clinical trial to test
the efficacy of a program to decrease developmental delays in low birth weight,
premature infants, measured participants’ experience of the early intervention (Liaw et
al., 1995). One of their measures tracked exposure to the intervention over the three year
trial period, using two frequency indicators: total number of days the child attended the
Child Development Center and total number of home visits received by the family.
Greater exposure to the program accounted for a small but significant amount of the
variance in IQ scores at 36 months of age.

The psychotherapy research field also tracks dosage frequency in their efforts to
understand the dose-response relationship for treatment benefits (Hansen, Lambert &
Forman, 2002; Howard, Kopta, Krause, & Orlinsky, 1986). Research indicates that the
more sessions a mental health consumer receives, the greater his/her improvements in
mental health outcomes (Hansen, Lambert & Forman, 2002; Howard, Kopta, Krause, &
Orlinsky, 1986).

In this study, a measure of training frequency at the coalition-level was tracked by
counting up the number individuals from a coalition that had attended more than one

CADCA training within the study period. A coalition that sends its members and staff to

52

LL

additional trainings may represent a coalition that is invested in continuous learning, not
just one shot learning opportunities for its staff. This may be one way to measure
continuous Ieaming, allowing the comparison of more active coalition learners (i.e.,
coalitions sending more members to additional training) to less active coalition learners
(i.e., coalitions sending fewer members to additional trainings).
Intensity of Exposure

Measures of intensity of exposure have typically been used as indicators of the
quantity of time an individual spends in an intervention within a specified period of time,
such as hours/week, days/month, weeks/year, etc. (Chaput et al., 2004; F iester et al.,
2005). Research suggests that higher intervention intensity is related to a variety of
outcomes. For example, higher intensity of attendance in after school academic programs
and programs to prevent aggressive behavior in children is associated with better
outcomes, including decreases in aggressive behaviors and higher word comprehension
scores (Anderson-Butcher, Newsome, & Ferrari, 2003; Chaput et al., 2004; F iester et al.,,
2005). Intensity of exposure has also been used in media campaigns to assess changes in
attitudes and behaviors of target groups. For example, VERB is a health marketing
campaign to help increase physical activity among children (Huhman, Potter, Duke,
Judkins, Heitzler, & Wong, 2007). Children were placed into five exposure groups based
on intensity of exposure (i.e., no exposure, exposure less than once a week, exposure
about once a week, exposure several times a week, and daily exposure). The findings
indicate a dose-response relationship based on intensity of exposure. More exposure was
linked to both stronger beliefs about the benefits of physical activity and higher self-

reported physical activity levels.

53

Applying an intensity measure to coalition training requires a twist on the
traditional ways this dosage measure has been used with individuals. While an
individual’s level of attendance in an after-school program or a prevention program may
be tracked by hours per week or sessions per year, coalition training is more infrequent
and intermittent. Training intensity may be better conceptualized by the average number
of individuals attending coalition trainings. While one coalition may send a single
individual to a training, another may send a group, with the expectation greater
representation at the training will result in greater benefit from the training. This measure
has contextual relevance. Many of the Institute trainings strongly encourage that multiple
coalition members attend the training together so that they may work together to apply
the training materials in small group work. Other trainings require teams of two
individuals to be co-trained side-by-side. The utility of this practice was tested out in this
study, determining if more individuals from a coalition attending a training provide added
benefit to the coalition.

Examining Multiple Measures of Dose

There is a small body of literature that suggests added benefit can be gained by
combining different measures of dosage into one analysis to determine if certain
combinations of dose better predict coalition outcomes. Reviews of the OST program
literature suggest that understanding of the dose-response relationship is strengthened
when multiple types of dose are examined in combination (Chaput et al., 2004; Fiester et
al., 2005). Combining these measures of dosage may provide a better understanding of
variability in coalition outcomes because coalitions naturally vary in how many

individuals are trained within a coalition, how many trainings they attend, and the

54

average number of individuals that attend a training. Given this variability a more
integrated approach to understanding coalition dosage is needed.

An evaluation of the After-School Corporation’s After School Program (TASC)
examined program duration and intensity (Welsh, Russell, Williams, Reisner, & White,
2002). The evaluators examined youth attendance by how long they had participated (0, l
or 2 years) and how actively they had participated (i.e., highly active, active or
nonactive). The evaluators assessed if duration and intensity were associated with better
reading and math scores. There was no relationship between attendance and reading
scores; however, highly active youth involved in the program for two years exhibited the
greatest increases in math scores and school attendance. The next highest math gains
were found in youth who were active for two years, followed by youth who were active
for one year. No significant gains in math scores were found in youth who were
nonactive participants. This suggests that dosage is a multi-element construct, and a
more accurate understanding of the relationship between outcomes and dose requires a
comprehensive, multi-faceted operationalization of dosage.

These findings suggest that greater richness of understanding occurs when
examining various combinations of exposure related to outcomes. In fact, when only
attuning to one aspect of attendance, findings may be obscured (Chaput, Little, & Weiss,
2004). There appears to be benefit in examining multiple measures in a single study to
look at the effect of training more comprehensively. This study continued this line of
inquiry by engaging in exploratory methods to determine how different combinations of
training dosage related to different levels of coalition capacity for community problem

solving.

55

 

Calculating Coalition Training Dosage

Most dosage work has been conducted at the individual level (Chaput et al.,
2004). However, in this study, dosage is conceptualized and operationalized at the
coalition or organizational level by adding up individual-level participation in training
and creating a coalition-level dosage score. By taking a concept used primarily at the
individual-level and applying it to the group level, dosage theory is expanded beyond its
original conceptualization. This assumes that training received by individuals of a
coalition can be aggregated to a group level and represent accurately a coalition’s
combined total dosage of training received. A potential problem with this approach may
be that individual-level dosage may not increase in an additive fashion when aggregated
but may increase multiplicatively or exponentially. Knowing how to weight individual-
level dosage to get an accurate group-level dose is not yet possible since very little
coalition training research has been conducted on even taking an additive approach to
dosage. There is some suggestion from the CTC training study that adding up individual-
level involvement in training to create a group dosage score is a suitable way to examine
this construct and its relationship to coalition-level training outcomes (F einberg et al.,
2002). The current study expands upon lessons learned in the CTC study and provides a
starting point of building a greater understanding of training dosage at the group level.

One potential bridge between individual-level dosage theory and the coalition
level of analysis may be in organizational learning theory. A key aspect of organizational
learning is that the only way for the organization to learn and grow is through the
individuals that make up the organization. “Organizations learn only through individuals

who learn. Individual learning does not guarantee organizational learning. But without it

56

no organizational learning occurs (Senge, 1990, pg. 129). One of the core disciplines in
organizational learning theory is that of team learning. Team learning speaks to the
process of alignment of the skills, tasks, energies and focus of the individual members of
an organization into a single, united direction. There is a systems perspective to this
conceptualization — these individuals work in choreographed synchrony together. By
learning together and across the organizational entity, the actions of the coalition can be
more coordinated and more effective.

Training offers coalitions an opportunity to learn and achieve this team-level of
functioning critical to a learning organization. Coalitions, like all organizations, are
made up of individuals. While individuals are the units that receive training, training is
designed to improve the work of the coalition — to make the coalition smarter, more
effective, and turning it into a learning organization (Butterfoss, 2007). Only through the
development of the skills and knowledge of individuals can the coalition increase its
group IQ, work in better collaboration with each other and put tools and resources to
better use. The CTC training study suggests that the more individuals trained in the
coalition, the more knowledge becomes institutionalized within the coalition to help
improve functioning (Feinberg et al., 2002).

The dosage types selected here offer a study into different mechanisms by which
training can impact the coalition’s leaming culture. An examination of the number of
unique individuals trained is one way to operationalize how widespread training occurs
across the coalition. If a coalition has only trained a single individual, then organizational
learning theory would suggest that team learning is less likely to occur, in comparison to

a coalition that has sent many different members to training (Senge, 1990). Furthermore,

57

this may indicate that policies and practices are not supportive of training, a condition
which has been included as a key dimension of a continuous learning orientation in
organizations (Tannenbaum, 1997). By examining training frequency as operationalized
by the number of individuals that have attended more than one training, lessons learned
from training can be honed and further practiced by the organization. This is one way
aspect of continuous learning, which has been found to be a key ingredient in predicting
the quality of products and services provided by non-profit organizations (McHargue,
2003). Finally, training intensity, or the average number of individuals from a coalition
typically being trained together, may foster the team learning that is a critical aspect of a
learning organization. A study of learning in non-profit organizations indicated that
organizations that encouraged collaborative learning and working were better able to
achieve their missions (McHargue, 2003). It appears then that these dosage measures are
potential ways to examine a coalition’s investment in learning. The corresponding results
of these investments were the key focus of the current study.

Additionally, all of these dosage types, when they are higher in amount, may also
promote diffusion of the learning. Diffusion of innovation theory suggests that the
innovation (i.e., community problem solving theory) may spread more quickly and
achieve an organizational tipping point when there is greater or more effective
communication about it within the members of the coalition entity (Rogers, 2003).
Diffusion of the training lessons occurs when the training information is shared and more
and more key opinion leaders within the coalition buy into the community problem
solving framework and are engaged in community problem solving behaviors. Even

though not all the key opinion leaders in a coalition may be able to attend a training,

58

according to this theory, if diffusion happens well, the untrained ones will benefit from
the training information because the knowledge and skills are shared with them (Rogers,
2003). Higher amounts of training, whether it is in terms of the number of individuals
trained, the number of additional training the individual receives or the average
attendance at training, may result in better diffusion of the training lessons across the
coalition because it means that information from the training is shared and used by more
people, is consistently communicated across the coalition, and spreads more quickly
throughout the organization. This diffusion facilitates the development of the learning
culture in that greater participation in training fosters a shared understanding of the role
of the coalition and opportunities to practice what has been taught in training. This grasp
of the “big picture” has been associated with aspects of a strong leaming environment,
including policies and practices that support training (Tannenbaum, 1997).

In summary, when conceptualizing coalitions and the impact of training on their
capacity for community problem solving, the most useful level of measurement appears
to be at the coalition-level (Feinberg et al., 2002). Since the Institute has trained
continuously on the elements of its community problem solving model since its
inception, it offers a rich context in which to examine dose-response for coalition training
and outcomes. Since there are no established dosage definitions for coalition training,
three types of coalition dosage are compared in this study to determine their usefulness at
predicting community problem solving capacity: breadth of training exposure, frequency
of training exposure, and intensity of training exposure. This study contributes to the
existing body of knowledge on coalition community capacity building by comparing the

predictive quality of these three types of dose as they relate to aspects of coalition

59

capacity for community problem solving. Based on organization learning theory, this
study hypothesized that coalitions with higher training frequency are more likely to see
better outcomes. Learning theory suggests that those coalitions with continuous
investments in learning opportunities (i.e., training) will be more successful because
through training they continue to learn and grow as organizations, constantly expanding
knowledge and skills (Senge, 1990).

HYPOTHESIS: Training frequency will be the strongest predictor of each
element of community problem solving.

In addition, combining types of dosage to better understand the relationship
between training attendance and elements of community problem solving capacity is
examined. Exploratory analysis examined the interaction between training breadth and
fiequency, allowing a more comprehensive and multi-faceted examination of the dose-
response relationship. Building upon learning organization theory (Senge, 1990),
coalitions that spread out learning across staff and members and also seek out training on
a frequent basis seem to value staff development (Hendry, 1996), an important element of
a learning priority. These coalitions may have the highest levels of coalition capacity for
community problem, exceeding levels predicted by a single dosage type.

PROPOSITION: Coalitions with both high training breadth and frequency will
have the highest levels of improvement in capacity for each of the three elements of
community problem solving.

Modeling the Dose-Response Relationship
The previous section clearly indicates that training participation dosage can and

should be defined and measured in a variety of ways. The next critical step is to

60

understand why and in which ways dose matters. While studies suggest that more
attendance is associated with better outcomes, there are variations in the dose-response
relationship that require additional consideration. There are three models that describe
how training may be associated with improved capacity for community problem solving
— linear model, curvilinear model and threshold model (Chaput et al., 2004; F einberg et
al., 2002; Holcomb et al., 1999).

Linear Model

Simply stated, the linear model suggests that outcomes increase as attendance
increases and that there are no limits to the benefits of attendance — more is always better.
Most of the studies in the Chaput et a1. (2004) review of the literature examined the linear
relationship between attendance and outcomes. Many of these studies suggest that
attendance intensity is positively related to outcomes, such as better school performance,
reductions in problem behaviors and emotional adjustment (Anderson-Butcher et al.,
2003; Youniss, McLeIlan, Su & Yates, 1999)

Many program evaluators have assessed the linear relationship between program
participation and program outcomes (August et al., 2004; Charlebois et al., 2004). The
Early Risers’ Skills for Success Program is a prevention program for youth at-risk for
developing conduct problems (August et al., 2004). The researchers found a linear
relationship between levels of participation (i.e., total number of hours of attendance in
the program) and outcomes (i.e., teacher-rated child behaviors, school adjustment, social
competence, externalizing and internalizing problems). Charlebois and colleagues (2004)
examined the dose-response relationship for a longitudinal prevention program for

disruptive boys. A linear relationship was discovered between program attendance and

61

program outcomes; indicating that the program was better for those who attended more
sessions.

The CTC studies (F einberg et al., 2002; Quinby et al., 2008) also found a linear
relationship between training attendance and CTC coalition capacity and functioning.
Additional research testing the linear relationship between training dosage and coalition
outcomes would be informative, especially to determine if a linear relationship describes
the dose-response for other types of dosage, such as frequency and intensity of training
exposure.

This linear model holds promise for coalition training, suggesting that continued
investment in trainings produces ongoing benefits for the coalition and its efforts.
Organizational learning theory provides some basis for why a linear model may best
describe the dose-response relationship. Organizations can increase their capacity to
succeed in an ever-changing world by creating mechanisms for continuous organizational
learning, or embracing an adoption of a learning organization (Snyder & Cummings,
1998). Using this perspective, organizational performance and learning are linked, and
value is placed in continuous personnel development (Hendry, 1996). Senge (1990)
describes an ideal type of organization that has the ability to incorporate learning into its
underlying values and beliefs. Senge describes an entity called a learning organization
that:

will be able to deal with the problems and opportunities of today, and

invest in its capacity to embrace tomorrow, because its members are

continually focused on enhancing and expanding their collective awareness

and capabilities. . .in other words, an organization which can learn (Senge, 1990,

62

p. 4).
Learning never ends, it is a long-term, continuous process necessary for organizational
success. Based on this framework, continued investments in learning are always
beneficial. Applied to training, greater investment in training, as measured by more
individuals trained (breadth of exposure), involvement in more training opportunities
(frequency of exposure), and more individuals trained per training (intensity of
exposure), is likely to be associated with coalitions that exhibit stronger capacity for
community problem solving. This model was tested in this study, examining if the
relationship between each type of dose and each element of community problem solving
is linear.

C urvilinear Model

The curvilinear model proposes that more moderate levels of attendance result in
better outcomes than higher levels of attendance (Chaput et al., 2004). In this model, low
attendance and high attendance produce less favorable outcomes. Those that receive a
lower dose may not receive a sufficient dose to benefit from the program. Those at a high
dose are at a disadvantage because they may be too engaged in the intervention or in the
training and neglecting other critical activities/responsibilities. Alternatively, there may
be an increase in Ieaming up to a point, but once a limit is met, additional training results
in no additional gains.

Some studies in the out-of-school time research arena suggest a curvilinear
relationship between intensity of attendance and youth outcomes; however, it may not be
a perfect inverted U-shaped curve. Instead, the association between the two variables may

suggest that while a moderate dose is the most advantageous, higher levels of dosage

63

produce better outcomes than lower dosage of attendance. This relationship is depicted
in studies of intensity and academic/career aspirations, school test scores and peer
relationships (Chaput et al., 2004). While youth who attended with high levels of
attendance did better than those with low levels of attendance, youth who attended at
moderate levels of attendance had the strongest improvements in their academic
achievement and relationships with others. However, a curvilinear relationship has not
been found for other types of exposure. Applied to training, this may mean that moderate
levels of training dose are associated with the most favorable outcomes. However, those
that receive high levels of training have better outcomes than those with low levels of
training.

Another interpretation of the curvilinear model can be applied from the fields of
biology, pharmacology and toxicology (Calabrese and Baldwin, 2001). In these fields, a
u-shaped dose-response relationship is associated with benefits at lower doses and
harmful effects at higher doses. For coalitions, this may mean that lower training dosage
is beneficial to a coalition because it provides the most beneficial and cost-effective use
of resources and time; while higher levels of training dosage impede coalition
functioning, perhaps because coalitions are overwhelmed with the amount of information
they have received. However, in order to fully capture a curvilinear dose-response
relationship, extensive longitudinal data need to be collected so that the full relationship
between dose and outcomes is described by the data and interpreted correctly (Calabrese
and Baldwin, 2001).

A curvilinear dose-response relationship has not been rigorously tested by

researchers. However, research in the adult Ieaming field also is suggestive of a potential

64

nonlinear relationship between involvement in adult Ieaming opportunities (i.e.,
employer-provided learning, academic courses, vocational courses, and leisure courses)
and health and social variables (Feinstein & Hammond, 2004). In this study, the most
favorable outcomes were associated with highest levels of participation in adult Ieaming,
except for smoking cessation and political cynicism. Involvement in more than 3 to 10
classes did not increase effects of Ieaming on smoking cessation, and effects for political
cynicism indicated no additional benefits after taking a second course. While the authors
did not provide potential reasons for these findings, it does appear that the relationship
between dose and outcomes may not always be linear.

Research from training transfer also suggests that the benefits of training may not
always be linear when examining the successful use of training to improve employee
performance in the organization (Baldwin & Ford, 1988). An individual may have
gained some skills immediately as a result of training and use those skills once back in
the workplace; however, skill use drops off over time perhaps because there is no support
in the work environment for using new skills. Additionally, it may take some time for an
individual to be able to put into place the skills they learned, so they may perform
“business as usual” until over time performance improves. Applied to coalition-level
training and dosage, it is possible that the benefits of additional training may not be
realized if the coalition environment does not support its staff and members in the use of
new skills learned in training. Additionally, training benefits may wane over time. Given
the great deal of work required to function well as a coalition and engage in efforts that
result in population-wide changes in targeted problems, it is important to know if the

benefits of continued training dry out and potentially do more harm than good. Research

65

is needed to fully understand if this model applies to coalition training and provide
insights into what levels of training are related to diminishing returns on investment.
Threshold Model

The threshold model is the simplest of the three models describing the
relationship between dosage and outcomes (Chaput et al., 2004). The basic tenet of this
model is that a minimum level of attendance is needed for individuals to experience a
positive result. This model also suggests similar results for individuals whose attendance
is at or beyond the threshold (Durlak & DuPre, 2008).

The threshold model has been used in other fields to understand the dose-response
relationship. Quercia and Galster’s (2000) review of the literature on neighborhood
change, suggests that change at this level does not unfold in a linear fashion. Instead,
empirical evidence exists to support the theory that neighborhoods change when
thresholds are met. For example, neighborhoods may change their racial composition
from primarily Caucasian to primarily a community of color when tolerance thresholds
for interracial living are met — this tipping point then shifts the rate of out-migration of
Caucasians and in—migration of individuals and families of color (Schelling, 1971 as cited
in Quercia & Galster, 2000).

The threshold model leads to a discussion of where and how thresholds should be
set. Looking to the established literature, however, does not provide many guidelines for
standardized thresholds or the process to estimate one. Quercia and Galster (2000)
cautioned that though thresholds existed, there did not appear to be one universal
threshold that could be applied to every neighborhood, suggesting each neighborhood

would have a unique threshold for change in neighborhood racial composition. Howard

66

and colleagues (1986) attempted to determine a threshold for psychotherapy treatment,
examining at what point a psychotherapy consumer should be included in a “treated”
group. Their review of the literature found that most studies indicated that it took six to
eight sessions for mental health consumers to see some positive change, so a person was
considered “treated” if they met this minimum threshold. Hansen and colleague’s (2002)
more recent review of the literature suggests that 57.6% to 67.2% of psychotherapy
consumers show improvements after approximately 13 sessions. However, they caution
that most consumers never meet the minimum threshold typically needed for therapeutic
benefit.

While the threshold model has relevance for coalition training in that it may
require a certain minimum amount of training to change coalition process and outcomes,
this was not tested in this study. Determining and detecting a nonlinear relationship, such
as a threshold, requires a large range and large sample (May & Bigelow, 2005), and the
dosage data available in this study do not lend themselves to assessing a threshold for
training amount. This study focused on testing the dose-response relationship for the
linear and curvilinear models, which was more feasible with the types of dosage data
available for examination.

Conclusion

Previous research provides the strongest evidence of a potential linear relationship
between training dosage and coalition outcomes (Chaput et al., 2004; F einberg et al.,
2002). Additionally, evidence from the Ieaming organization literature suggests that
coalitions benefit from investments in continuous Ieaming (Hendry, 1994; Senge, 1990).

Coalitions that participate in more training may have infused a Ieaming orientation into

67

their culture. Considering the constantly changing world of community-based prevention
and shifting priorities of funders and policy makers, coalitions need to be adaptive to new
situations. Learning is one way to be adaptive, and so it was hypothesized that coalitions
that received more training dosage did better in each element of the capacity for
community problem solving model.

HYPOTHESIS: A linear relationship will exist between each type of training
dosage and each component of community problem solving capacity.

Coalition Age as a Moderator of Training Dosage and Coalition Outcomes

Proponents of the coalition stages of development model propose that coalitions
naturally develop overtime, suggesting that older coalitions may have greater capacity
since they have sustained themselves and have had greater opportunities to engage in
coalition work related to their goal (Butterfoss, 2007; Butterfoss, Goodman &
Wandersman, 1993; Florin, Mitchell & Stevenson, 1993). The national evaluation of the
Drug Free Communities Support Program (DFC) also suggests that DFC coalitions are
maturing over time and that the more mature coalitions are more likely to have lower
rates of 30-day use of alcohol, tobacco and marijuana in comparison to less mature
coalitions (ONDCP, 2008). It is possible, then, that coalition age may moderate the
relationship between training dosage and coalition capacity for community problem
solving. Coalitions may take advantage of training very differently based on their age.
For example, an older coalition may require less training than a younger coalition in that
it has already built up capacity through time and experience. It is also possible that older
coalitions may require greater amounts of training in order to break coalition “bad

habits”. This study included an exploratory analysis of coalition age as a potential

68

moderating variable that may change the relationship between the training dosage and
coalition outcomes.

PROPOSITION: The amount of training a coalition needs to build coalition
capacity for community problem solving will be related to the age of the coalition.

Additional Considerations in Dose-Response Modeling —- Controlling for Covariates

Research on coalitions suggests that other factors besides training may impact
coalition capacity for community problem solving, such as coalition budget, size and
grant funding status. These require careful attention in any coalition study.

Coalition capacity for community problem solving may be impacted by coalition
budget. While coalition budget does not appear to be the defining factor for coalition
success (Wolff, 2001), coalitions do require some monetary resources to operate and
maintain their coalition and its work and need to put attention into infusing financial
resources into their coalition (Butterfoss, 2007; Foster-Fishman et al., 2001). As such, it
is likely that coalitions with larger budgets are able to attend more trainings than
coalitions with smaller budgets since they have more discretionary funds to do so or a
greater number of funding sources that require training participation.

Coalition size is also likely to be a covariate, especially regarding the number of
individuals trained in a coalition, with larger coalitions more able and likely to send more
individuals. As such, this was an additional factor covaried out in the analyses.

Finally, Drug-Free Communities (DF C) grant funding status was another potential
confound that could explain the relationship between training and outcomes. As part of
the grant application, coalitions must demonstrate they have engaged in a community

assessment, include a logic model/strategic plan as part of the application, and indicate

69

how they will evaluate their efforts (Department of Health and Human Services RF A No.
SP-O9-002). The Institute Evaluation findings corroborate that DF C coalitions have more
planning products in place, use them more frequently, and are engaged in more of the
essential processes than non-DF C coalitions (F oster-Fishman et al., 2008). This
evaluation also indicated that the Institute’s training and technical assistance was relevant
for DFCs, triggering the same community problem solving process in DFCs and non-
DFCs. However, in order to examine training dosage’s unique contribution to changes in
community problem solving capacity, DF C funding status was covaried out.
Current Study

The purpose of this study was to examine the dose-response relationship between
coalition training and elements of coalition capacity for community problem solving:
strong internal functioning, use of comprehensive strategies, and facilitation of
community change (see Figures 3-5). This study used data fiom CADCA’s training
participant database and its Annual Survey of Coalitions to examine if there were
differential impacts of training based on different dosages of training. The study
conducted analyses to determine if training improved coalition outcomes, which type of
training dosage was most predictive of coalition outcomes, and how the relationship

between dosage and outcomes should be defined.

70

Figure 3. Dose-Response Relationship for Training Dosage and Coalition Internal
Functioning

 

Breadth of Exposure

 

 

 

 

Coalition Internal
Functioning

 

 

Frequency of Exposure

 

 

 

 

 

 

\/

 

Intensity of Exposure

 

 

 

Figure 4. Dose-Response Relationship for Training Dosage and Comprehensiveness of
Strategies

 

Breadth of Exposure

 

 

 

 

Comprehensive
Strategies

 

 

Frequency of Exposure

 

 

 

 

 

 

 

\/

Intensity of Exposure

 

 

 

Figure 5. Dose-Response Relationship for Training Dosage and Facilitation of
Community Change

 

Breadth of Exposure

 

 

 

 

Facilitating
Community
Change

 

 

Frequency of Exposure

 

 

 

 

 

 

\/

 

Intensity of Exposure

 

 

 

7l

The following research questions were examined:

1. Does training improve coalition capacity for community problem solving?

2. What is the nature of the relationship between training dosage and elements of
capacity for community problem solving?

3. Which type of training dosage received (i.e., breadth, frequency or intensity) is
the strongest predictor of each element of capacity for community problem
solving?

Research Question #1: Does training improve coalition capacity for community problem
solving?

HYPOTHESIS: Coalitions that have received training from the Institute will be
stronger in each element of community problem solving capacity than coalitions not
trained by the Institute.

Training appears to be a critical element to help coalitions build needed skills and
capacities to successfully address local substance abuse issues (F einberg et al., 2002;
Florin et al., 1993). Training has been shown to increase knowledge and skills.
Additionally, Institute trainings appear to be beneficial to coalitions in that coalitions
trained by the Institute develop and use more coalition planning products (F oster-
Fishman, Law & Ahn, 2008). Coalitions trained by the Institute also report becoming
more effective as a result of training, making better choices regarding selecting
appropriate strategies to implement, and expanding involvement (Foster—Fishman, White,
& Droege, 2007). As such, it was hypothesized that coalitions that have received training
fiom the Institute are stronger in each element of community problem solving capacity in

comparison to those not trained by the Institute.

72

Research Question #2: What is the nature of the relationship between training dosage and
elements of community problem solving capacity?

HYPOTHESIS: A linear relationship will exist between each type of training
dosage and each component of community problem solving capacity.

There appears to be stronger evidence of a linear relationship between dosage and
outcomes based on research in other fields (Chaput et al., 2004; Feinberg et al., 2002).
Additionally, the Ieaming organization framework proposes organizations infuse a
continuous learning culture to improve organizational functioning and success (Senge,
1990). Based on this, a linear relationship between training dosage and coalition capacity
for community problem solving was hypothesized, indicating that greater training
exposure is associated with better coalition outcomes.

Research Question #3: Which type of training dosage (breadth, frequency or intensity)
received is the strongest predictor of each element of community problem solving
capacity?

HYPOTHESIS: Training frequency will be the strongest predictor of each
element of community problem solving.

Three different measures of training dosage were compared in this study: training
breadth (i.e., total number of individuals trained), training frequency (i.e., number of
individuals that have attended more than one training) and training intensity (i.e., average
coalition attendance at trainings.) It was hypothesized that training frequency would be
the best predictor of coalition capacity for community problem solving. Again, drawing
from the learning organization literature, coalitions that continue to seek out Ieaming

opportunities may have a strong Ieaming orientation (Senge, 1990). The number of

73

individuals attending additional training is one measure of continuous Ieaming and it was
hypothesized that this dosage type would have the strongest relationship to coalition
outcomes.

Proposition #1 : Coalitions with both high training breadth and frequency will have the
highest levels of improvement in each of the three elements of community problem
solving capacity.

Studies suggest that the combination of different dosage types offers a more
nuanced understanding of what comprises active participation in training and that
outcomes may vary due to this interplay (Chaput et al., 2004). In addition to comparing
the different predictive characteristics of each training dosage type, this study explored if
coalitions with high levels of training for two types of dosage (breadth and frequency)
have greater capacity for community problem solving.

Proposition #2: The amount of training a coalition needs to build coalition capacity for
community problem solving will be related to the age of the coalition.

The coalition stages of development theory proposes that coalitions go through a
maturation process with the most mature coalitions having the greatest capacity to
address their local substance abuse issues (Florin et al., 1993). Based on this theory,
coalition training may have different relevance to coalitions based on their age. This
proposition was examined through exploratory analysis to determine if coalition age
moderates the relationship between training dosage and coalition capacity for community

problem solving.

74

CHAPTER 3: RESEARCH DESIGN AND METHODS
Overview
In order to investigate the proposed hypotheses, this study used existing data
collected by CADCA and its National Coalition Institute. Because training is designed to
build coalition skills and capacity in each element of CADCA’s community problem
solving model described in the previous section, a study examining training impact needs
to assess change over time. A pre test/post-test study design was used to examine how
coalitions improve as a result of differing levels of coalition training exposure. Since it is
possible that coalitions that seek out more training may have greater existing capacity,
Time 1 scores for measures of coalition community problem solving capacity were
covaried out to examine how training dosage types predict Time 2 scores. Approval for
secondary data analysis was obtained from Michigan State University IRB.
Procedures
Setting
The study examined a national sample of coalitions trained by CADCA’s
National Coalition Institute. This offered an appropriate context for this study for a
variety of reasons. Most coalition studies have used case studies or examined coalitions
involved in a single initiative (i.e., Fighting Back coalitions, CSAP Community
Partnerships). By using a national sample untied to any particular coalition firnding
stream, there was a greater likelihood of generalizability outside of the coalitions
included in this study. In addition, this setting provided rich data on Institute training

history for each coalition and data collected for two time points on coalition capacity for

75

community problem solving, allowing the examination of multiple types of dosage and
investigating changes in coalition outcomes over time.
Sample

The sample consisted of 269 substance abuse coalitions that had participated in
CADCA’s Annual Survey of Coalitions, a comprehensive survey of coalitions. Of these,
182 had received training from CADCA while 87 coalitions served as a no-training
control condition. Pre-test data were collected in 2005 for 43% (n=115) of the sample and
in 2006 for the remaining 57% (n=154) of the sample. Post-test data were collected in
the 2008 administration of the survey. This sample allowed comparison of those
coalitions who had received training to those that had not. In addition, the sub-sample of
182 coalitions with training allowed an in-depth examination of training dosage targeted
in this study.

Coalition demographic information at Time 2 is provided below in Table 4 for the
dosage, untrained and total samples. The total sample was fairly evenly distributed
across budget, with 38% reporting budgets of less than $100,000, 35% with budgets
ranging from $100,000 to $199,999, and 27% reporting budgets $200,000 and above.
Mean age of the total sample was 11.02 years (SD = 6.45). The age of the total sample
also had good spread across age categories: 21% 0-5 years, 33% 6-10 years, 23% 11-15
years, and 22% 16+ years in age. There was greater representation from communities
targeting counties (50%) and working in rural (54%) and suburban (29%) settings. In
addition, the coalitions in the total sample considered themselves to be fairly mature, with

56% self-reporting intermediate level of maturity and 37% self-reporting being at the

76

advanced level. Most of the total sample (71%) currently received DFC funding, and
most had 1-2 staff members (66%).

Comparisons of the dosage and untrained sub-samples do indicate differences
among groups. Chi-squares were performed to determine if these coalitions varied on the
covariates of budget, size and funding status. The dosage sample had larger budgets [x2
(2, N=182) = 107.52, p<.001], more staff [x2 (3, N=182) = 8.21, p<.05], and was more
likely to have DFC funding than the untrained sample [x2 (I , N=182) = 96.31, p<.001].
In terms of other demographic characteristics, the dosage and untrained sub-samples were
not significantly different in age, t(250) = -.37, p = .713. Mean age for the dosage sub-
sarnple was 11.12 years (SD = 6.34) and mean age for the untrained sub-sample was
10.80 years (SD = 6.71). The trained and untrained coalitions appeared to be similarly
distributed in terms of their geographic target areas [x2 (3, N=182) = 1.27, p = .74] and
degree of focus on ATOD issues [x2 (1, N=182) = 1.81, p = .18]. However, the two sub-
samples did vary in terms of perceived maturity [x2 (2, N=182) = 8.17, p<.05], with the

dosage group rating themselves as more mature.

77

Table 4. Sample Demowphic Characteristics

 

 

 

Dosage Untrained Sub- Total
Sub-sample sample Sample
(11:1 82) (IT-'87) (0:269)
Characteristic % % %
Budget
Less than $100,000 26 63 38
$100,000 to $199,999 42 20 35
$200,000 and above 31 17 27
Geographic Target Area
County 49 51 50
City 15 14 15
School District 1 I 9 '10
Description of geographic target area
Frontier 3 2 3
Rural 53 55 54
Suburban 29 29 29
Urban 15 I4 15
Age
0-5 years 18 27 21
6-10 years 36 27 33
11-15 years 24 23 23
16+ years 22 23 22
Self described level of maturity
Novice 5 ll 7
Intermediate 56 56 56
Advanced 39 33 37
Focus on Alcohol, Tobacco and Other Drugs
(ATOD)
Mission focused solely on Alcohol, 61 S6 60
Tobacco and Other Drug Issues (ATOD
Specific) .
ATOD issues as part of a broader 39 44 41
mission/array of issues
Number of Full-Time Equivalent Staff
None 8 16 ll
Fewer than 3 58 52 66
3-5 24 22 23
More than 6 10 10 10
DFC Grant Status
DFC Grantee 82 46 71

 

78

Power to Detect Differences

Most coalition studies have been conducted with small samples or individual case
studies (F awcett et al., 1997; Paine-Andrews et al., 2002). The use of small sample sizes
in many evaluations of community-based prevention efforts, including coalitions, has
been criticized since lower statistical power exists to detect differences (Merzel &
D’Affitti, 2003). In this study, power was calculated using GPower for multiple
regression with 3 predictors. A sample size of 182 coalitions achieves a power of .70 to
detect an effect size of .05. This effect size is between the small (r-squared = .02) and
medium (r-squared = .15) effects, but coalition studies tend produce small effects (Allen,
2001; Nowell, 2005).

Measurement Development and Data Collection
Training Dosage

One of the biggest challenges associated with dose-response analyses is in
tracking and accessing accurate records that track intervention use. Many researchers
emphasize the need to implement systems to track attendance (Dane & Schneider, 1998;
McCall, Ryan, & Plemons, 2003; Valentine et al., 1998). The two processes frequently
used to capture this information are self-report (Anderson-Butcher et al., 2003; Basen-
Engquist, O’Hara-Tompkins, Lovatto, Lewis, Parcel, & Gingiss, 1994) and assessment of
attendance/service utilization records (Bickman et al., 2002; F einberg et al., 2002;
Quinby et al., 2008). Some studies have looked at recall of participants, asking them if
they have utilized the program or services. However, the study of a Comprehensive
Child Development Program (CCDP) compared participant recall with records in a

Management Information System (MIS), and the results were very different (McCall,

79

Ryan, & Plemons, 2003). While the evaluators said no evidence existed to fully know
which method would provide more reliable information, they suggested that the self-
report method of semi-annual recall of services used in the CCDP seemed to be more
problematic than weekly case manager entries into the MIS system. Regardless of the
method, in order for dose-response to be assessed, dosage data must be tracked
consistently.

CADCA’s Institute has kept a database of all the coalitions and the individuals
within the coalition that it has trained since its inception. Training attendance was
collected through paper/pencil sign-in sheets at each training. Participants were required
to complete the sign-in, and the information was entered into an Excel database
maintained by CADCA Institute staff. The following information was collected for each
training: training name, date, individual participant name, coalition name and contact
information, trainer name and location of the training. Dosage for each dosage type for
each coalition was calculated for the period between Time I and Time 2.

Coalition Capacity for Community Problem Solving

CADCA’s Annual Survey of Coalitions (Annual Survey) was used to measure the
dependent variables in this study: coalition internal functioning, use of comprehensive
strategies, and coalition facilitation of community change. The Annual Survey measures
a variety of coalition characteristics, including community sector involvement, budget,
geographic target area, use of the essential processes, use of comprehensive strategies,
and coalition promotion of community changes for substance abuse. CADCA’s Annual
Survey was created to serve as coalition census tool so that the organization could gain a

better understanding of the number of anti-drug coalitions in America, demographic

80

 

characteristics of coalitions, what coalition processes they were engaged in, and what
substance abuse prevention activities they were involved in. The Survey was developed
by staff at CADCA’s Institute who first conducted a literature review to understand
qualities that defined coalition effectiveness, coalition typologies, and survey instruments
commonly used in coalition evaluations.

CADCA also convened a Technical Advisory Committee (TAC) comprised of
various coalition researchers to review the literature summary and to provide guidance to
CADCA in the development of a coalition survey. The committee recommended key
topical areas that should be included in a coalition survey. The Institute used TAC
member feedback and their review of existing coalition surveys to develop the Annual
Survey of Coalitions. The Survey tool was then reviewed by the TAC and also by
coalition leaders. Two focus groups were convened with coalition leaders to review the
Survey and provide feedback on content and flow. The focus groups were each
comprised of approximately 10 coalition leaders representing coalitions of various size,
age and geography. The coalition leaders also varied in how long they held a leadership
position within the coalition. The final survey draft was programmed to be taken online,
and the online version of the survey was reviewed by the same focus group participants
and by coalitions that were members of CADCA’s Coalition Advisory Committee, a
governance committee comprised of coalition and state substance abuse leaders.

The Annual Survey of Coalitions was first launched in February, 2005 and
coalitions were able to participate during a four month field period until June 2005. 887
coalitions participated in that survey administration. The Annual Survey was repeated in

2006 with 699 coalitions participating that year during a 4 month field period that began

81

 

in February and ended in June.

Between Fall 2006 and Spring 2007, the Institute reviewed the Annual Survey
tool to determine how well it could be used to evaluate the Institute’s services and
supports to the coalition field. Specifically, the Institute desired to use the Annual Survey
to better understand the effectiveness of its training and other supports. Since the Annual
Survey’s reach was nation-wide, it could be used to track a large number of coalitions
over time. The Institute, in close consultation with its external evaluator, revised the
Annual Survey to better serve as an evaluation tool. Toward that end, additional items
were added to measure the Institute’s theory of change (Figure 2), including items to
measure each element of its community problem solving model. The Survey
administration also moved from the beginning of the year to the end of the calendar year,
to better align with the Institute’s fiscal year, which runs October l-September 30m.
Many coalitions are also on the same fiscal schedule and since the Survey asks coalitions
to answer questions on activities conducted in the previous 12 months, the 4-month
Survey field period was now better timed to match coalition and Institute reporting
calendars.

The Annual Survey is an online survey, and one survey is completed for each
coalition. Instructions strongly encourage that the coalition leader be the primary
respondent for the coalition since he or she is the individual most likely to have the
deepest understanding of the coalition and its efforts. Annual Survey participants agree
to participate by checking a box that signifies they understand how Annual Survey data
will be used and shared. They are aware the information may be used for research

purposes but know that individual-level identifying information will not be shared as part

82

of the study. A paper/pencil version of the survey is available for coalitions without
intemet access; however, the majority of coalitions take the survey online. For example,
only 6% of respondents in 2007 completed the paper/pencil tool. CADCA also provides
incentives to coalitions to participate in the Annual Survey. Coalitions that complete the
survey are entered into a drawing to win prizes such as free conference and training
registration, Ipods and digital cameras.

The Annual Survey has undergone revisions from the 2006 to the 2008 version,
and while constructs are operationalized somewhat differently for each year, they are
each measuring the same latent construct and thus does not prohibit using the Annual
Surveys for pre/post comparisons. A description of each of the constructs and how they
were measured each year are provided in Appendix A.

Measures
Training Dosage

Training dosage was calculated for each coalition for training received between
pre and post-test. Training dosage scores were calculated for any training received up to
one month before the coalition participated in the 2008 Annual Survey. While a few
coalitions have received training even a week prior to participating in the 2008 Survey, a
month cut-off was established for inclusion in the dosage calculation, providing some
time in which coalitions can implement the knowledge and skills learned in training.
Training dosage was operationalized in three ways for this study and dosage scores for
each coalition were calculated using data from training attendance records.

Breadth of Exposure

Breadth of exposure was calculated by totaling the number of unique

83

t.

individuals trained in each coalition. If one person attended more than one
training, he or she was only counted once.
Frequency of Exposure

Frequency of exposure to training was calculated by summing the number
of individuals from each coalition that had attended more than 1 training. For
example, if a coalition trained 5 members, but one individual attended an
additional training from CADCA, regardless of the topic, the coalition was given
a score of 1.
Intensity of Exposure

Intensity of exposure was calculated by summing the number of
individuals from the coalition that attend each training and then dividing this
number by all the training events the coalition has attended. This measure differs
from the training breadth measure in that the intent is not to know how many
unique individuals were trained but rather the average number of individuals a
coalition sends to trainings. For example if the summed total attendance across 3
trainings was 9 people, then the coalition had sent an average of 3 people to each
training, indicating it tended to send more than one individual to Institute
trainings.

Coalition Capacity for Community Problem Solving

Coalition capacity for community problem solving was measured with data

collected in CADCA’s Annual Survey of Coalitions. The three elements comprising

capacity for community problem solving (i.e., internal functioning, comprehensiveness of

84

strategies and community change agent status) and how they were measured are
described below.
Coalition Internal Functioning
Four elements comprise coalition internal functioning: l) coalition use of essential
processes; 2) development and use of quality planning products; 3) active coalition
membership involvement; and 4) coalition climate supportive of collaboration.
Coalition essential processes. Coalition use of essential processes was measured
by subscales that assessed if a coalition has implemented activities related to the seven
essential processes covered in the Institute trainings. These items were measured
categorically (yes/no), with the coalition indicating whether or not they had been
involved in each of the activities in each subscale. For each subscale, all the items in
which the coalition indicated they were involved were totaled. Each subscale was
averaged by dividing the number of active items with the total number of items in the
subscale to create an individual subscale average. A description of the seven subscales
and items under each are described below. The 2008 Survey subscales had more items
than the 2005/2006 Survey, expanding upon items collected in the earlier Surveys.
(Please see items 20, 27-29 in Appendix B; items 21, 28-30 in Appendix C; item 34 in
Appendix D.
0 Assessing community needs and resources — In the 2005/2006 Annual Survey,
this subscale consisted of two items. The 2008 subscale had 7 items.
Example items include: Collected data to determine or monitor the extent of
substance abuse problems in the community; Collected data on risk and

protective factor predictors for substance abuse and other related problems.

85

Analyzing information about the problem or goal -— This subscale had one
item in the 2005/2006 Annual Survey and four items in the 2008 Survey.
Example item include: Used data to prioritize substance abuse needs;
Identified local conditions (personal and environmental) that maintain risk or
build protection in your community

Developing a fiamework of change — This subscale had one item in the
2005/2006 Annual Survey and four items in the 2008 Survey. Examples of
items include: Developed a logic model and objectives for what the coalition
will accomplish; Ensured the logic model reflects the culture and values of the
community

Developing and using strategic and action plans — Three items comprised this
subscale in both the 2005/2006 Annual Survey and 2008 Annual Survey.
Sample items include: Developed a written strategic plan to reduce substance
abuse in their target area; Developed population-level strategies/activities for
changing community conditions and behaviors.

Identifying, adapting and implementing interventions — The 2005/2006
subscales were comprised of two items, and the 2008 subscale was comprised
of five items, such as Matched prioritized substance abuse needs with
evidence-based programs/strategies; Developed local or “homegrown”
programs/strategies to address prioritized substance abuse needs.

Evaluating the coalition — The 2005/2006 subscales contained seven items
while the 2008 subscale contained 11 items: Collected data to assess

immediate/intermediate outcomes of programs/strategies; Disseminated

86

program evaluation results and monitoring data to key community
stakeholders.

Sustaining projects and initiatives — The 2005/2006 Annual Survey consisted
of a one-item subscale and the 2008 Survey consisted of a seven-item
subscale. Example items include: Used data as a basis for new grants or

funding proposals; Identified what must be sustained.

Development and use of quality planning products. This scale measured coalition

product development/revision, product use and the quality of each product (i.e.,

Community Assessment, Logic Model, Strategic/Action Plan, Evaluation Plan and

Sustainability Plan). Planning product questions were only measured in 2008. A total

Planning Products score was created by adding up the three subscale scores (please see

items 35, 36 and 38 in Appendix D):

Product development/revision - The Institute’s training model emphasizes that
products are not fixed and should be constantly revised and updated.
Coalitions were asked if they had developed or revised each of the five
products in the last 12 months. A subscale score is created indicating the total
number of products the coalition has developed or revised in the last 12
months. Subscale scores range between 0-5.

Product use — Coalition respondents rated how often they used each of the five
planning products to inform the decisions of the coalition. The five point
Likert scale ranged from 1 = “not at all” to 5 = “always”. The coalition could

also indicate if they did not have the product. A total product use score

87

indicating how often they used the five products was created by taking the
mean of the use score across all 5 products.

0 Product quality - Respondents were asked to rate the degree to which each
product (if the coalition has created one) was a comprehensive, effective
planning tool for the coalition. The four-point Likert scale ranged from 1 =
“not at all” to 4 = “a great deal”. A product quality score indicating the total
quality of the five products was created by averaging the quality scores across
all of the 5 planning products.

Active coalition membership. Coalitions were provided a list of
sectors/organizations often found represented on substance abuse coalitions. Coalitions
were asked to select which groups are represented on their coalition. Then, coalitions
were asked about the quality of participation, measuring how actively the organization
participated on the coalition. In the 2005/2006 Annual Survey, coalitions indicated
membership participation in three areas: strategy/program development, strategy/program
implementation, and resources (e.g., space, funding, people, administrative, operational,
programmatic supports). Active membership was operationalized as the proportion of the
number of total membership that provide support in at least two out of three areas.
(Please see items 2 and 3 in Appendix B and C).

In the 2008 Survey, coalitions used a four-point Likert scale ranging from 1 to 4
(1 = “not at all” to 4 = “a great deal”) to rate each sector/organization’s contribution to
the coalition’s strategic efforts. Active membership was operationalized as the
proportion of sectors/organizations in the total membership with contribution ratings of 3

= “somewhat” or 4 = “a great deal.” (Please see Items 2 and 3 in Appendix D).

88

Coalition climate supportive of collaboration. Coalition climate was measured
with a five-item scale. This sub-scale was only measured in Time 2. Each item was
measured on a 4 point Likert scale items ranging from 1 = “strongly disagree” to 4 =
“strongly agree.” Examples of items in this scale are: coalition members share a common
vision for the community, the coalition has an effective conflict resolution process, and
the coalition uses collaborative decision-making processes (see item 9 in Appendix D).
Responses were averaged to create a coalition climate for collaboration scale score.
Psychometric analysis was conducted to explore the internal consistency of this scale.
The 4-item climate for collaboration scale had an alpha of .82 for the total sample
(Dosage sub-sample 01 = .82, Untrained sub-sample a = .82).

Comprehensiveness of Strategies

This construct measured the implementation of the coalition’s strategic plan,
assessing the degree to which each coalition was engaged in a comprehensive set of
behavior change strategies that are likely to bring about widespread community change.
(See item 15 in Appendix B and C and Item 39 in Appendix D)

Total strategy comprehensiveness. In the 2005/2006 Annual Survey, coalitions
answered a set of dichotomous items that listed common interventions to address
substance abuse. These interventions were organized into six behavior change strategies:
Providing information, Enhancing skills, Providing support, Enhancing access/reduce
barriers, Changing consequences, and Modifying/changing policies. Coalitions indicated
if they had been involved in any of the interventions in the last year. If a coalition had
been involved in any activity under one of the six strategies, then the coalition was scored

a one for that category. In the 2008 Annual Survey, this scale was measured differently

89

than in Time 1. An additional behavior change strategy of “Changing the physical design

of the environment” was included so that 7 behavior change strategies were assessed in

Time 2. Each of the seven strategies was described and coalitions indicated the extent to

which they were engaged in the strategy (4-point Likert scale, ranging from “not at all” to

“a great deal”). For both Time 1 and Time 2, behavior change strategies were aggregated

into two types of strategy categories: program and environmental change strategies.

Total Program Strategies - The first three behavior change strategies (providing
information, enhancing skills and providing support) are more individually-
focused strategies used in substance abuse prevention programming. For Time 1,
a total program strategy score was calculated for each coalition by counting up
each program-focused strategy the coalition had been involved in, ranging from 0
to 3. For Time 2, a total program strategy score was calculated by counting the
number of program-focused strategies in which the coalition indicated it was
involved (not very much, somewhat, a great deal). Each coalition was given a
score from 0 to 3. For both Time 1 and Time 2, a higher number indicated greater
involvement in program strategies

Total Systems Strategies - The remaining behavior change strategies (enhancing
access/reducing barriers, changing consequences, modifying/changing policies
and changing the physical design of the environment) are considered to be more
environmentally focused strategies. In Time 1, a total environmental strategy
score was calculated for by counting up each environmental strategy the coalition
has been involved in, ranging from 0 to 3. In Time 2, a total environmental

Strategy score was calculated by counting the number of environmentally-focused

90

strategies in which the coalition indicated it was involved (not very much,

somewhat, a great deal). Each coalition was given a score from 0 to 4. For both

Time 1 and Time 2, a higher score indicated greater involvement in

environmentally-focused change strategies.

Average strategy involvement. Average strategy involvement scores were also
calculated, broken down into program and environmental categories. In the 2005/2006
Annual Survey, as described above, coalitions indicated if they had implemented specific
interventions (e.g., information dissemination campaign, community development
activities, alternative activities, and enforcement of community policies/laws) associated
with each of the 6 broad change strategies. For each strategy, the number of
interventions implemented for each change strategy was summed. In the 2008 Annual
Survey, the ratings for each of the seven behavior change strategies (0 = “not at all” to 4
= “a great deal”) were used to calculate average strategy involvement.

0 Average Program Strategy Involvement — In Time 1, an average program strategy
score was calculated by meaning the three strategies comprising this subscale
(providing information, enhancing skills and providing support). In Time 2, the
score was calculated by meaning the ratings across the three strategies.

' Average Environmental Strategy Involvement — In time 1, an average
environmental strategy score was calculated by meaning the three strategies
comprising this subscale (enhancing access/reducing barriers, changing
consequences and modifying/changing policies). In Time 2, the score was
Calculated by meaning the ratings across the four strategies (enhancing

access/reducing barriers, changing consequences and modifying/changing

91

policies, and changing the physical design of the environment).
Facilitating Community Change

Most documentation of community changes have been done with tracking and
assessing each community change facilitated by the coalition (Allen et al., 2008; Collie-
Akers et al., 2007; F awcett et al., 1997; Paine-Andrews et al., 2002). While feasible in
individual case studies or studies with small samples, the Annual Survey has hundreds of
participants and is not an appropriate method for gathering specific community change
information for each participant. Instead, the Institute consulted with community change
experts to determine how to best measure this in a large scale survey. While individual
community changes could not be tracked via the Annual Survey, what could be measured
wais if coalitions had helped bring about new or modified programs and
policies/practices, in what sectors, for what targets, and the degree to which these
changes aligned with the mission and goals of the coalition.

The Annual Survey measured if coalitions have facilitated both new or modified
programmatic changes and policy/practice changes. The Institute strived to incorporate
as much of the KU measurement of community change as was appropriate in a large-
scale survey. Survey respondents were also asked to indicate the type of change they
helped bring about (program or policy/practice), in which sectors they were able to bring
about the programmatic and policy/practice changes (i.e., nonprofit, business,
government/law, education/school, community, health, social services, media, family and
caregiver, religious and youth), who the targets of the changes were (i.e., youth, parents
and caregivers, families, community leaders, and the general community), and the extent

to which these changes matched the coalition’s targeted outcomes. These items were

92

used to measure if coalitions were engaged in change strategies in multiple sectors and
for multiple targets, Iwith more widespread change indicating that coalitions had a large
circle of influence. The following aspects related to facilitation of community change
were measured:

0 Types of change: Coalitions were asked if they had brought about change in
new/modified programs and change in new/modified policies or practices.
For each type of change, the coalition was given a score of 0=no or 1=yes,
indicating if the coalition had facilitated the type of change.

9 Expanded circle of influence

0 Policy/Practice change sectors: the number of sectors in which they
were able to bring about a policy/practice change (scores range 0-11)

0 Program change sectors: the number of sectors in which they were
able to bring about a program change (scores range 0-11)

0 Policy/Practice targets of change: the number of targets of change for
policy/practice change (scores range 0-5)

0 Program targets of change: the number of targets of change for
programmatic change (scores range 0-5)

0 Change is appropriate to local context — Three items assessed the degree to
which the community changes the coalition had helped facilitate addressed
targeted issues and are appropriate to the local context were also measured.
Each item was measured on a 4 point Likert scale (“not at all” to “a great

deal”). Ratings were averaged to create a subscale score. The 3-item scale

93

had an alpha of .98 for the total sample (Dosage sub-sample 01 = .98,
Untrained sub-sample 01 = .99). (See items 48-50 in Appendix B)

0 Changes are strategic — This variable was created by creating a rating of
whether the changes the coalition had brought about in last 12 months were
strategic. Coalitions indicated if the changes they had helped facilitate
addressed targeted substance abuse problems, community risk/protective
factors and local conditions that contribute to substance abuse. Coalitions
were rated 0-3, with a higher number indicating more strategic changes.

Coalition Descriptive Information

The Annual Survey also included questions pertaining to demographic
characteristics of the coalition, such as its age, budget, geographic target area description,
funding sources and staff and volunteer size. Time 2 scores for budget, size and funding
status were covariates in this study. For funding status, a large percentage of Annual
Survey respondents were coalitions with Drug Free Communities Support Program
grants. As part of the application process, these coalitions needed to submit a logic
model, evaluation plan and discuss the community assessment process used to gather data
to justify how they will use grant funding. Since these coalitions may have had higher
levels of capacity because of their grant application requirements, DFC grant status was

covaried out.

94

CHAPTER 4: RESULTS

To conduct the analyses in this study, a pre-post database was created where 2005
and 2006 pre-test survey data were merged with 2008 post-test survey data. Dosage data
were added for the subset of coalitions that had received training between Time 1 and
Time 2 (n=182) and an external party removed all identifying information for the
purposes of this study.

Outliers

Potential outliers and influential observations were identified by examining
bubble plots that plotted the studentized residuals with leverage values. Points were
scaled by Cook’s D measure. Bubble plots were examined for all of the three dosage
measure IVs against each of the three community problem solving DVs. In no instances
were cases identified as being both unusual (i.e., outlier) and having undue influence on
the model. All observations were kept in the sample.

Univariate and Descriptive Analyses

Descriptive statistics including frequencies and measures of central tendency were
used to examine the sample distribution for each variable of interest in this study.
Additionally, correlations were conducted to examine the relationships among the
variables.

Measures of central tendency and other descriptive statistics on each of the three
types of training dosage of interest in the study for the 182 coalitions that received
training are provided in Table 5. In addition, the correlation among each of the three
dosage types is provided in Table 6. The three IVs are all slightly positively skewed, and

there does appear to be some range restriction in that not many coalitions are in the high

95

range of training exposure for any of the three dosage types. However, examination of
the histograms for each of the IVs does indicate that the distributions do approach
normality.

Table 5. Descriptive Statistics on Training Dosage

 

 

 

 

 

 

 

Breadth of Exposure Frequency of Exposure Intensity of Exposure
(Number of Individuals (Number of individuals (Average Number of
Trained) attending more than 1 Individuals Attending
trainirg) Each Training)
Mean 3.96 1.17 1.93
Median 3 .00 1.00 1.85
Mode 2.00 0.00 2.00
Standard Deviation 2.97 1.33 .88
Variance 8.83 1.77 .78
Minimum 1.00 0.00 1.00

 

Maximum 24.00 8.00 6.00

Table 6. Correlations between Dosage Types

 

 

 

 

Training Breadth Training Frgquency Training Intensity
Training Breadth l
Trainirg Frequency .61 * 1
Traimngintensity .61 * .22 * l

 

* Correlation is significant at the 0.01 level.

The independent variables were highly correlated with each other (See Table 6).
Multicolinearity diagnostics were performed to determine if the strong correlation
between predictor variables would inflate the sampling variances for the coefficients.
Multicolinearity was measured by examining the variance inflation factor (VIF), which
measures how much a predictor increases the variance of the coefficients. If the predictor
has a VIF > 10, then this is problematic. The VIFs for all the predictors were less than 10,
indicating that the high correlation among independent variables was not problematic.

Missing Data Imputation

A Missing Values Analysis using SPSS 17.2 was performed to determine the

extent of missing data in the dataset. Sub-scales to be used in computing the principal

components for all Time 1 and Time 2 measures of coalition capacity for community

96

problem solving and covariates were used in the Missing Values Analysis. The original
pre-post dataset had 303 cases. However, missing data analysis indicated that 34 cases
were missing at least 75% of data in either Time 1 or Time 2. Differences between
response and non-response were examined to see if missingness varied by coalition
demographic characteristics. Chi-squares were performed to determine if these 34
coalitions varied on the covariates of budget, size and fimding status in comparison to
coalitions will less than 75% missing data. Overall, it appears that the coalitions with
missing data do not vary from the non-missing group in terms of budget, staffing or DF C
Status. The 34 cases were removed from the dataset and multiple imputation was used
to impute data for the remaining 269 cases. Data were imputed for 29% of the remaining
sample. Of these imputed cases, most (88%) had missing data for less than 10% of the
variables of interest in this study.

Table 7. Bivariate Relationships of Demographic Characteristics with Missing Data
Status

 

 

 

 

 

Missing Data Status
Chi-Square Analyses % (n) Sig.
Coalitions Coalitions with
with more less than 75%
than 75% missing data
missing data (n=269)
(n=34)
Covariates
Budget
Less than $100,000 35 (10) 38 (103) _
$100,000 to $199,999 35 (10) 35 (94) 12 (2,3422; -71
$200,000 and above 31 (9) 27 (72) (P ‘ ' )
Number of F ull-Time Equivalent Staff
None 10(3) 11 (29)
Fewer than 3 66 (19) 56 (150) x2 (3, 34) = 4.03
3—5 10 (3) 23 (63) (p = .291)
More than 6 I4 (4) 10 (27)
DFC Grantee Status
Yes 59 (20) 71 (190) x2 ((1; 3413—2337
No 41 (14) 29 (79) '

 

97

Little’s MCAR test was used to determine if data were missing completely at
random. The significant chi-square test indicates that data are not missing completely at
random, x2 (1163, N=303) = 1444.29, p<.001. However, multiple imputation is robust
against violations of MCAR and is a highly regarded approach to imputing missing data
(Tabachnick & F idell, 2007). Since the imputation process selects values for missing data
at random, it preserves not only the sample size but also improves any inferences to be
made of analyses using the imputed database (McKnight, McKnight, Sidani & F igueredo,
2007; Schafer & Graham, 2002). Ten imputations were selected and all analyses
discussed further were conducted using the imputed dataset. Unless otherwise indicated,
all results report pooled data, which averages the statistics for the original and the 10
imputed datasets.

Outcome Measure Scale Construction

The dependent variables were Time 2 scores on the three elements of community
problem solving of interest in this study: coalition internal functioning,
comprehensiveness of strategies and facilitation of community change. Since the three
outcome measures and the scales used to construct them were developed for this study, to
create an overall measure of each outcome variable at Time 1 and Time 2 each outcome
measure (coalition internal functioning, comprehensiveness of strategies, and facilitation
of community change) was constructed using principal components analysis. The first
principal component, which summarizes the scales and items into one variable containing
most of the variance for that latent construct, was used in each analysis. This offered a
parsimonious way to create one variable combining multiple scales that are comprised of

both continuous and categorical variables (Tabachnick & F idell, 2007).

98

A description of how each of the principal components was extracted for each
Time 1 and Time 2 measure of coalition capacity for community problem solving is
described below. Also, it should be noted that while Time 1 and Time 2 are not on the
same exact scale, it is a similar sort of scaling that measures the same construct. Using
the principal component allows the comparison of constructs that were not measured
identically in Time 1 and Time 2. Since Time 1 scores are covaried out, it allows an
interpretation of Time 2 scores after controlling for initial levels of community problem
solving. This is a standard way of conceptualizing pre-post designs.

Descriptive statistics for the subscales comprising each Time 1 and Time 2
Principal Component Analysis are reported in Tables B and C in Appendix A. An overall
correlation matrix between the [VS and composite DVs is located in Table D in Appendix
A.

Internal Functioning
Time 1

To create the lntemal Functioning Scale, the following variables were entered into
the principal component analysis: Assessing community needs and resources, Analyzing
information about the problem or goal, Developing a framework of change, Developing
and using strategic and action plans, Identifying, adapting and implementing
interventions, Evaluating the coalition, Sustaining projects and initiatives, and Proportion
of active coalition members. Only one component with an eigen value greater than 1 was
extracted. This component accounted for 40% of the variance in the dataset (see Table E

in Appendix A for the total variance explained).

99

Time 2

The following variables were entered into the principal component analysis:
Assessing community needs and resources, Analyzing information about the problem or
goal, Developing a framework of change, Developing and using strategic and action
plans, Identifying, adapting and implementing interventions, Evaluating the coalition,
Sustaining projects and initiatives, Proportion of active coalition members, Total number
of products developed, Average product quality, Average product use, and Quality of
Collaboration. Three components with eigen values greater than 1 were extracted,
accounting for 66% of the total variance in the dataset (see Table F in Appendix A for the
total variance explained). The first component accounted for 47% of the total variance
and appeared to be most highly correlated with the essential processes. The second
component accounted for 10% of the total variance and appears to be most correlated
with planning product development. The third component accounted for another 9% of
the total variance and was most highly correlated with quality of membership (Table J in
Appendix A for the component matrix). Examination of the scree plot indicated that the
first component provided the best summary of the variables included in the analysis.

Comprehensiveness of Strategies

Time 1

The following variables were included in the principal component analysis:
average involvement in programmatic change strategies, average involvement in
environmental change strategies, number of programmatic strategies and number of
environmental change strategies. A single component was extracted that accounted for

62% of the total variance among the variables (Table G in Appendix A).

100

Time 2

Four variables were included in the principal component analysis: Average
involvement in programmatic change strategies, average involvement in environmental
change strategies, number of programmatic strategies and number of environmental
change strategies. As in Time 1, only 1 component with an eigen value greater than 1
was extracted, accounting for 62% of the variance (Table H in Appendix A).

Facilitation of Community Changes

Time 1 data were not collected for this construct. A principal component analysis
was conducted on the following Time 2 variables: If facilitated any new/modified
program changes, if facilitated any new/modified policy/practice changes, number of
sectors influenced by program changes, number of sectors influenced by policy/practice
changes, number of targets influenced by program changes, and number of targets
influenced by policy/practice changes. Two components were extracted, accounting for
76% of the total variance (see Table I in Appendix A for total variance explained). The
first component accounted for 57% of the variance among the variables in this analysis
and was mostly highly correlated with the changes fitting the mission of the coalition and
being strategic. The second component accounted for approximately 20% of the total
variance and was most highly correlated with whether or not the coalition had brought
about a policy or practice change (see Table K in Appendix A for the component matrix).
While examination of the scree plot did not show significant drop-off alter the first
component, the 37% difference in variance accounted for between the two components
did indicate that the first component offered the most significant summary of the

variables in this construct.

101

Survey Year

The dosage sub-sample of 182 coalitions consisted of 54% with pre-test data
collected in 2005 (n=99) and 46% with pretest data collected in 2006 (n=83). Chi-square
tests were performed to determine if these sub-samples varied on key demographic
variables. Results of the chi-square tests are provided in Table 8. The chi-square
analyses indicate that the 2005 and 2006 sub-samples are fairly equal distributed in terms
of budget, staffing, DF C status and ATOD focus. The two sub-samples do appear to vary
in terms of geographic target area and self-described level of maturity, with the 2005 sub-
sample more mature [x2 (2, 99) = 10.15 (p < .001 )] and more rural in their make-up [x2

(3, 99) = 23.02 (p<.01)].

Table 8. Bivariate Relationships of Demographic Characteristics with Baseline

Survey Year

 

Baseline Survey Year

 

Chi-Square Analyses % (n) Sig.
2005 2006
(n=99) (n=83)
Demographic Characteristics
Budget
Less than $100,000 23 (23) 3O (25) _
$100,000 to $199,999 44 (44) 40 (33) 7‘2 (f’ :9)3;)2'18
$200,000 and above 32 (32) 30 (25) p '
Number of Full-Time Equivalent Staff
None 6 (6) 11 (9)
Fewer than 3 57 (56) 59 (49) x2 (3, 99) = 6.09
3-5 25 (25) 23 (19) (p = .11)
More than 6 12 (12) 7 (6)
DFC Grantee Status
Yes 81 (80) 84 (70) X2 (1, 99) = .77
No 19(19) 16(13) (p=.41)
Focus on ATOD
Mission focused solely on Alcohol, 62 (61) 60 (50)
Tobacco and Other Drug Issues =
(ATOD Specific) XZ ((1:39I4)'1 1
ATOD issues as part of a broader 38 (38) 40 (33) p '
mission/array of issues

102

Table 8 (cont’d).

 

Self described level of matruity
Novice
Intermediate
Advanced

Description of geographic target area
Frontier
Rural
Suburban
Urban

2 (2)
53 (52)
45 (44)

3(3)
61 (60)
18 (18)
18 (18)

9 (7)
59 (48)
33 (27)

4 (3)
45 (37)
41 (34)

11 (9)

x2 (2,99) = 10.15
(p = .000)*

x2 (3, 99) = 23.02
(p = .006)"

 

*Significant at p < .001
”Significant at p < .01

Further analyses were performed to determine if an additional year to implement

the training lessons for the 2005 subsample made a difference in how these coalitions

took advantage of training. A binary variable indicating which group the coalition is in

(Time 1 = 2005 Survey / Time 1 = 2006 Survey) was added to test if this variable

interacted with dosage, indicating that dosage had different effects based on how long the

coalition had to implement the training lessons. Hierarchical multiple regression was

used to test if cohort status had a main effect and interaction effect with training dosage

on Time 2 outcome variables.

Only coalitions with training dosage were used in this analysis (n=182). Cohort

status was dummy coded with the 2005 group serving as the reference group. The

categorical covariates of coalition budget, size and DFC fimding status were dummy

coded. Coalition budget categories were collapsed into three categories. Budget

categories for $100,000-$ 199,999 and over $200,000 were entered into the model, and

the less than $100,000 category was excluded as the reference group. Coalition size was

also dummy coded so that variables representing 1-2 staff, 3-5 staff, and 6 or more staff

were entered into the model while the no staff category was excluded as the reference

group. For DFC firnding status, the funded group was entered into the model.

103

The variables were entered into the model in blocks, with Time 1 community
problem solving scores and covariates in block 1, training dosage and dummy coded
2006 survey year in block 2 and the interaction of the training dosage variable and survey
year. Nine separate regressions were performed, examining each of the 3 dosage types’
relationship to each of the 3 outcome variables.

In none of the analyses did cohort status have a main effect on the post-test
outcome variable. However, in one, this variable did have a significant interaction with
dosage. The interaction between survey cohort and training frequency (number of
individuals attending more than one training) significantly predicted Coalition lntemal
Functioning, (B: .24, t (171) = 2.45, p<.05). The multiple R for the regression model
with covariates was significant, predicting 6% of the total variance in Coalition lntemal
Functioning, (F (7, 174) =5.35, p<.001). R2 Change with the addition of the training
frequency and survey cohort year in the second model was not significant. However, the
third model with the interaction between survey cohort and training frequency did

increase the variance accounted for, (R2=.03, p<.05) (F (10, 171) = 4.83, p<.001).

104

Figure 6. Training Frequency by Survey Year

 

Training Frequency

 

 

2005

Survey Year

 

 

2006

 

Figure 7. Time 2 Coalition lntemal Functioning by Survey Year

 

.0
b)
1

0.25 A
0.2 -
0.15 —

O

'09
OMH
1 1

; -0.05 -
-o.1 «

Coahtron Internal Functionin;

 

+2005
- I - 2006

 

-0.15 .
Time]

Data Collection Period

 

Tim: 2

 

Examination of graphs (Figures 6 and 7) indicated that the 2005 sub-sample had

higher levels of training frequency and higher levels of Time 2 coalition internal

functioning than the 2006 sub-sample. These findings indicate that in comparison to the

2005 sub-sample, the 2006 sub-sample experienced a decrease in the rate of change in

lntemal functioning for every unit decrease in the number of individuals attending

additional training. In this case, it appears that the 2005 sub-sample was able to send

105

 

 

more individuals to additional training and also experienced better outcomes in coalition
internal functioning. Since an interaction existed between training frequency and survey
year when predicting Coalition lntemal Functioning, survey year was included as an
additional covariate in analyses when training frequency and lntemal Functioning were
examined.
Improving Statistical Power

Given the large number of predictors and covariates, this study is underpowered
to detect differences due to the intervention. As such, before final statistical analyses
were conducted to test each hypothesis, linear regressions were performed to examine the
relationship between each of the covariates (staffing, budget and DF C status) and each
separate dependent variable to determine covariates not playing a significant role in the
overall model. Regressions were run with the full sample and also the dosage sub-
sample. Tables 9-11 provide the results for each of the regression analyses. The
findings are discussed below.

Coalition Internal Functioning

Multiple R for both the full sample and dosage sample regression models
predicting Time 2 Coalition lntemal Functioning using only the covariates was
statistically significant (F (7, 261)=7.66, p<.001) and (F (7, 174)=5.35, p<.001)
respectively (Table 9). For both regressions, there were main effects for Time 1 lntemal
Functioning and coalition staffing size. Higher Time 1 lntemal Functioning and more
staffing was associated with higher levels of Time 2 Coalition lntemal Functioning. In
the remainder of analyses, DFC status and budget were excluded since they did not

significantly predict Time 2 Coalition lntemal Functioning.

106

Table 9. Covariates Predicting Time 2 Coalition lntemal Functioning

 

 

 

 

Full Sample Dosage Sample
(n=267) (n=182)
8 SE 13 (3 p B SE B p p
(Constant) -.549 .183 .003 -.758 .230 .001
”15"“ .1 “item“ .267 .060 0.24*** .000 .199 .067 0.19M .003
unctronrng
Have DFC Gram VS' .267 .137 0.11 .052 .113 .160 0.05 .482
no Grant
Budget $100,000-
$199,999 vs. Less .182 .145 0.08 .208 .257 .152 0.14 .091
than $100,000
Budget $200,000 and
above vs. Less than -.034 .155 -0.01 .829 -.014 .167 -0.01 .934
$100,000
1-2 staffvs. no staff .266 .192 0.12 .165 .658 .222 0.35" .003
3-5 staff vs. no staff .456 .224 0.18* .042 .879 .251 040*" .000
6 0' "‘0” “am" “0 .610 .259 017* .019 .975 .287 0.31M .001
staff
R2 017*" 0.18***

 

* P < .05; ** P < .01; ***P < .001

C omprehensiveness of Strategies

Multiple R for both the full sample and dosage sample regression models

predicting Time 2 Comprehensiveness of Strategies using only the covariates was

statistically significant, (F (7, 261)=6.93, p<.001) and (F (7, l74)=5.69, p<.001)

respectively (Table 10). For the full sample, there were main effects for Time 1

Comprehensiveness of Strategies, DFC status and budget. Higher Time 2

Comprehensiveness of Strategies was associated with higher Time I Comprehensiveness

of Strategies, having DF C funding and having budgets greater than $99,999. For the

dosage sample, there were main effects for Time 1 Comprehensiveness of Strategies and

DFC status. Here, higher scores in Time 1 Comprehensiveness of Strategies and

receiving a DFC grant were associated with higher scores in Time 2 scores in the DV.

107

The nonsignificant covariate of staffing was removed from the remainder of analyses
using the full sample, and budget and staffing were not included in analyses with the

dosage sample.

Table 10. Covariates Predicting Time 2 Comprehensiveness of Strategies

 

 

 

 

Full Sample Dosage Sample
(n=267) (n=182)
8 SE B (3 p 8 SE B (3 p
(Constant) -.465 .186 .012 -.637 .238 .007
Time 1
Comprehensive .223 .060 O.20*** .000 .221 .063 023*" .000
Strategies
Have DFC Gram VS' .324 .142 013* .022 .527 .166 0.21** .001
no Grant
Budget $100,000-
$199,999 vs. Less .453 .148 0.20" .002 .239 .155 0. 12 .122
than $ 100,000
Budget $200,000 and
above vs. Less than .359 .158 0.14“ .023 .106 .169 0.05 .529
$100,000
1-2 staff vs. no staff -.045 .195 -0.02 .817 .103 .228 0.05 .651
3-5 staff vs. no Staff .005 .226 0.00 .982 .242 .257 0.11 .346
6 0‘" mm Staffvs' “0 .159 .266 0.04 .551 .469 .294 0.15 .111
staff
R2 0.16*** 0.19***

 

* P < .05; ** P < .01; ***P < .001
Facilitation of Community Change
Multiple R for both the full sample and dosage sample regression models
predicting Time 2 Comprehensiveness of Strategies using only the covariates was
statistically significant (F (7, 26l)=3.79, p<.01) and (F (7, l74)=3.50, p<.05) respectively
(Table 11). For the full sample, there were main effects for budget and staffing, and for
the dosage sample, only staffing had a main effect. In the full sample, having a budget

ranging fiom $100,000-$ 199,999 and having staff were associated with higher Time 2

108

Facilitation of Change. In the dosage sample, higher Time 2 Facilitation of Change was

associated with having coalition staff. DF C status was not included in further analyses

using the full sample, and DFC status and coalition budget were removed from analyses

using the dosage sample.

Table 11. Covariates Predicting Time 2 Facilitation of Change

 

 

 

 

Full Sample Dosage Sample
(n=267) (n=182)
B SE B I3 p B SE B [3 p
(Constant) -.640 .193 .001 -.693 .272 .01 1
Have DFC Gram VS' -.035 .148 -002 .814 .006 .192 0.00 .976
no Grant
Budget $100,000-
$199,999 vs. Less .329 .156 0.15* .034 .292 .180 0.15 .104
than $100,000
Budget $200,000 and
above vs. Less than .212 .167 0.09 .203 .233 .197 0.11 .236
$100,000
1-2 staff vs. no staff .467 .205 0.23* .023 .542 .264 0.28”“ .040
35 staff vs. no staff .805 .237 0.34** .001 .841 .298 0.38** .005
6 °’ more Staffvs' “0 .631 .274 0.19* .021 .752 .340 023* .027
staff
R2 0.08** 008*

 

*P<.OS;**P<.01

Results of Hypothesis Testing

Research Question I .' Does training improve a coalition ’8 community problem solving

capacity?

T-test analyses were first conducted to determine if there were significant

differences between the trained and untrained groups at Time 1. This was done to

determine if coalitions with higher levels of community problem solving capacity sought

out training. There were no differences between the trained and untrained groups in

Time 1 Scores for any of the pre-test community problem solving scores, indicating that

109

these groups had similar scores at baseline (see Table 12).

Table 12. Bivariate Relationships of Training Status with Time 1 Elements of
Community Problem Solving Capacity

 

Training Status

 

 

Mean Comparisons M (SD) Sig.
Untrained Trained
(11:87) (n=182)

Coalition lntemal Functioning -.06 (1.12) .16 (.91) I: '1-71

(df = 267, p = .087)

t = -.29

Comprehensiveness of Strategies .02 (1.00) .06 (.98) (df= 267 p = 775)

Hierarchical multiple regression analyses were used to examine if training status
predicted the DVs once Time 1 scores and covariates were controlled for. Coalitions
were divided into two groups: trained (n = 182) vs. not trained (n = 87) and training
status was used to predict each of the three dependent variables (coalition internal
functioning, comprehensiveness of strategies and facilitation of community change) in
this study. The categorical independent variable was dummy coded before it was entered
into the analysis, with the non-trained category used as the reference group, and three
separate regressions were run, one for each dependent variable.

For the covariates, only the statistically significant variables were included in the
regressions for each DV. Coalition staffing, budget and DF C status were dummy coded
before they were put into the model. Coalition budget categories were collapsed into
three categories. Budget categories for $100,000-$199,999 and $200,000 were entered
into the model, and the less than $100,000 category was excluded as the reference group.
Coalition size was also dummy coded so that variables representing 1-2 staff, 3-5 staff,
and 6 or more staff were entered into the model while the no staff category was excluded
as the reference group. For DFC funding status, the funded group was entered into the

model. The variables were entered into the model in blocks, with Time 1 community

110

problem solving scores and significant covariates in block 1 and the training status
variable in block 2.
Coalition lntemal Functioning.

Multiple R for the regression model predicting Time 2 intemal coalition
functioning using only the covariates was statistically significant (F (4, 264) = 10.87,
p<.001). The model predicted 14% of the variance in Time 2 coalition internal
functioning. Adding the training status variable significantly increased the amount of
variance explained (16%) in Time 2 Coalition lntemal Functioning (R2 Change=.02,
p<.05) (F (5, 263) = 10.06, p<.001). In comparison to coalitions that did not have
training, coalitions receiving training experienced higher Time 2 Coalition lntemal
Functioning after covarying out Time 1 Coalition lntemal Functioning and staff size.

Table 13. Training Status Predicting Coalition lntemal Functioning

 

 

 

 

Model 1 Model 2

Variable B SE B [3 p B SE B [3 p
(Constant) -.37 .18 .027 -.55 .19 .003
Tune .‘ 19‘3”“ .29 .06 0.28*** .000 .27 .06 0.27*** .000
Functioning
1-2 staffvs. no staff .35 .19 0.18 .063 .31 .19 0.15 .106
3-5 staff vs. no staff .57 .21 0.24" .008 .53 .21 0.22* .014
6 °’ “‘0“ Staff VS' “0 .66 .25 0.20** .009 .63 .25 0.19* .013
staff
Received Training .30 .12 0.14* .014
R2 0.14 0.16
F for changg in R2 10.88*** .000 598* 0.015

 

* P < .05; ** P < .01, *** P < .001

Ill

Comprehensiveness of Strategies

Multiple R for the regression model predicting Time 2 comprehensiveness of
coalitions strategies using only the covariates was statistically significant (F (4, 264) =
11.95, p<.001), predicting 15% of the variance. Training status did not significantly
increase the amount of variance accounted for in Comprehensiveness of Strategies,
indicating that training was not associated with Time 2 comprehensiveness of strategies
(see Table 14).

Table 14. Training Status Predicting Comprehensiveness of Strategies

 

 

 

 

Model 1 Model 2

Variable B SE B [3 p B SE B [3 p
(Constant) -.48 . 12 .000 -.48 . 12 .000
Time 1
Comprehensive .23 .06 0.23*** .000 .23 .06 0.22*** .000
Strategies
Have DFC Gram VS' .33 .14 0.15* .017 .34 .14 015* .020
no Grant
Budget $100,000-
$l99,999 vs. Less .44 .15 0.21" .003 .44 .15 0.21" .003
than $100,000
Budget $200,000 and
above vs. Less than .38 .15 0.17* .012 .38 .16 0.17* .014
$100,000
Received Training -.01 .14 -0.01 .917
R2 0.15 0.15
F for change in R2 I l.95*** .000 0.01 0.918

 

"‘ P < .05; ** P < .01, *** P < .001
Facilitation of Community Changes

Multiple R for the regression model predicting Time 2 facilitation of community
change using only the covariates was statistically significant (F (5, 263) = 4.55 p<.01),
predicting 8% of the variance. Training status did not significantly increase the R2 and

was not statistically significant in the prediction of Facilitation of Community Changes

112

(see Table 15).

Table 15. Trainig Status Predicting Facilitation of Community Change

 

 

 

 

Model 1 Model 2
SE SE
Variable B B B P B B B "
(Constant) -.65 .19 .001 -.70 .20 .000
Budget $ 100,000-
$199,999 vs. Less .32 .14 0.15* .028 .27 .15 0.13 .072
than $100,000
Budget $200,000
and above vs. Less .20 .16 0.09 .210 .16 .17 0.07 .336
than $100,000
1-2 staff vs. no staff .46 .20 0.23“ .023 .45 .20 0.22“ .027
3-5 staff vs. no staff .80 .23 0.33” .001 .79 .23 0.33M .001
3023;?” Staff VS' .62 .27 018* .022 .62 .27 0.18* .023
Received Training .12 .14 0.06 .382
R2 .08 .08
F for change in R2 4.55*** 0.001 0.76 0.383

 

* P < .05; ** P < .01, *** P < .001
Post-hoe Analyses

Post-hoe analyses were conducted to examine the differences between the trained
and untrained groups in Coalition lntemal Functioning. The trained group had a mean
Time 2 score of .16 (SD = .86) and the untrained group’s mean was -.22 (SD = 1.20).
Independent t-tests comparing the trained and untrained groups at Time 2 indicated that
the trained group had statistically significantly higher scores in Coalition lntemal
Functioning, t(267) = -3.02, p = .003.

Paired t-tests were used to examine if pre and post test scores for the trained and
untrained groups on Coalition lntemal Functioning were significantly different. The

results indicate that the trained group’s Coalition lntemal Functioning scores were stable

113

from Time I to Time 2, t(181) = .03, p = .976. The untrained group did experience
decreases in their pre and post test scores for Coalition lntemal Functioning but these
decreases were nonsignificant, t(86) = 1.19, p = .235. The pattern of response for both
groups is depicted in Figure 8.

Figure 8. Coalition lntemal Functioning by Data Collection Period and Training Status

 

0.2 1

0.15 . t A

 

0.1 1

0.05 4
0 - --e- - Untrained

-0.05 — N —I—Trained
-0.1 4 ‘ .
-O.15 . \ .

-O.2 . \ .
-0.25 we—e a

Tine 1 Time 2
Data Collection Period

 

 

 

 

Coalition lntemal Functionin:

 

 

 

 

 

 

Research Question 2: What is the nature of the relationship between training dosage and
elements of capacity for community problem solving?

To determine the nature of the relationship between each training dosage type and
each element of capacity for community problem solving, nine scatterplots were graphed,
examining each independent variable’s relationship to each Time 2 measure of the three
dependent variables. Linear, quadratic and cubic lines of best fit were drawn but none of
these were significant (Tables 16-18).

A visual inspection of the scatterplots was conducted. Lowess smoothing was
used to determine a line of best fit and examination of each of the 9 scatterplots suggested

that a linear relationship best describes the relationship between training dosage and

114

community problem solving capacity. The lowess smoothing line also best approximated
the linear line of best fit that was generated, providing a better graphical representation of
the data than the quadratic and cubic lines of best fit. The graphs of all the scatterplots
can be found in Appendix A, Figures A-I.

Table I6. Coalition Internal Functioning and Training Dosage: Curve Fit Model
Summary and Parameter Estimates

 

 

 

 

 

 

 

 

Model Summary Parameter Estimates
R
Equation Square F dfl df2 Sig. Constant b1 b2 b3
Training Breadth (# of Individuals Trained)
Linear 0.00 0.21 1 180 0.650 0.12 0.01
Quadratic 0.00 0.33 2 179 0.719 0.18 -0.02 0.00
Cubic 0.01 0.37 3 178 0.778 0.08 0.05 -0.01 0.00
Training Frequency (# of Individuals Attendigg More than One Training)
Linear 0.02 3.57 l 180 0.060 0.05 0.09
Quadratic 0.02 2.05 2 179 0.132 0.02 0.15 -0.01
Cubic 0.03 1.62 3 178 0.186 -0.01 0.29 -0.08 0.01
Training Intensity (Average Attendance at Trainings)
Linear 0.00 0.10 1 180 0.750 0.20 -0.02
Quadratic 0.00 0.28 2 179 0.760 0.39 -0.20 0.03
Cubic 0.01 0.47 3 178 0.701 -0.12 0.51 -0.25 0.03

 

115

Table I 7. Comprehensiveness of Strategies and Training Dosage: Curve Fit Model
Summary and Parameter Estimates

 

 

 

 

 

 

 

 

Model Summary Parameter Estimates
R
Equation Square F dfl df2 Sig. Constant b1 b2 b3
Training Breadth (# of Individuals Trained)
Linear 0.00 0.54 1 180 0.465 0.04 0.02
Quadratic 0.01 0.59 2 179 0.553 0.12 -0.02 0.00
Cubic 0.01 0.41 3 178 0.744 0.16 -0.04 0.01 000
Training Frequency (# of Individuals Attending More than One Training)
Linear 0.01 1.17 I 180 0.281 0.04 0.05
Quadratic 0.01 0.61 2 179 0.547 0.05 0.03 0.00
Cubic 0.01 0.49 3 178 0.688 0.04 0.12 -0.04 0.00
Training Intensity (Average Attendance at Trainings)

Linear 0.00 0.47 1 180 0.493 0.01 0.05
Quadratic 0.00 0.24 2 179 0.784 0.05 0.02 0.01
Cubic 0.01 0.71 3 178 0.547 -0.68 1.01 -0.40 0.04

 

Table 18. Facilitation of Change and Training Dosage: Curve Fit Model Summary and
Parameter Estimates

 

Model Summary

Parameter Estimates

 

 

 

 

 

 

 

R

Equation Square F dfl df2 Si g Constant bl b2 b3

Training Breadth (# of Individuals Trained)

Linear 0.00 0.08 l 180 0.777 0.12 -0.01

Quadratic 0.01 0.69 2 179 0.500 -0.00 0.05 -0.00

Cubic 0.02 1.02 3 178 0.383 0.23 -0.10 0.02 -0.00
Training Frequency (14 of Individuals Attending More than One Training)

Linear 0.00 0.14 1 180 0.710 0.12 -0.02

Quadratic 0.02 1.87 2 179 0.157 0.03 0.17 -0.04

Cubic 0.02 1.24 3 178 0.296 0.03 0.15 -0.03 -0.00

Trainingfltensity (Average Attendance at Trainings)

Linear 0.00 0.00 l 180 0.968 0.09 0.00

Quadratic 0.00 0.00 2 179 0.998 0.07 0.02 -0.00

Cubic 0.01 0.39 3 178 0.762 -0.59 0.94 -0.37 0.04

 

116

Research Question 3 .' Which type of dosage is the strongest predictor of each element of

capacity for community problem solving?

Correlations between each dosage type and outcome measures were first

examined to determine if the variables were related to each other. None of the dosage

measures were significantly correlated with any of the DVs. The correlation matrix

between the [VS and DVs is depicted in Table 19.

Table 19. Correlation Matrix of IVs and DVs (Dosage Sub-Sample)

 

 

1 2 3 4 5 6 7 8
1. Training Breadth ~-
2. Training Frequency .61 * --
3. Training Intensity .61“ .22* --
4. Coalrtrorr Internal . 08 . 0 6 .10 __
F unctromng Time 1
5. Comprehensiveness of *
Strategies Time I '03 '01 "02 '22 --
6' coal‘fm‘. Inter?“ .03 .14 -.03 .28* .18** --
F unctronrng Time 2
7. Comprehensiveness of * * t __
Strategies Time 2 .06 .08 .05 .24 .29 .54
8. Facrlrtatron of Community _.02 ‘03 .00 .18** .15“ ’44,, ’47,, __

Change Time 2

 

Note: * P < .01; ** P < .05.

Hierarchical multiple regression analyses were used to determine which type of

dosage (breadth, frequency or intensity) was the best predictor of each element of

capacity for community problem solving. The sub-sample of 182 coalitions with training

dosage was used for these analyses. Three separate regressions were conducted for each

of the three dependent variables. As in Research Question 1, only the significant

covariates were included in the analyses (See Tables 9-11). The variables were entered

ll7

into the model in blocks, with each Time 1 community problem solving score and the
covariates in block 1 and the three training dosage variables in block 2.

While the Multiple R for the regression models predicting each of the three
elements of community problem solving using only the covariates were statistically
significant, addition of the dosage variables into the models did not significantly increase
the variance explained. R2 did not significantly increase with the addition of the dosage
variables in block 2 for any of the analyses. Examination of the coefficients indicated
that training frequency did predict Coalition lntemal Functioning (B: .18, t(264) = 2.06,
p<.05); however, since the R2 change for this analysis was nonsignificant, training
frequency does not account for a significant portion of the variance in post-test Coalition
lntemal Functioning. All of the other analyses were nonsignificant in that in no other
case were any of the three types of training dosage associated Coalition lntemal

Functioning, Comprehensiveness of Strategies or Facilitation of Community Changes

(Tables 20-22)

118

Table 20. Training Dosage Predicting Coalition lntemal Functioning

 

 

 

 

 

 

 

 

 

Model 1 Model 2

Variable B SE B [t p B SE B [3 p
(Constant) -.47 .22 .030 -.48 .28 .086
gm": .1 18‘3”“ .22 .07 0.23** .001 .22 .07 o.23** .001

unctronrng
1-2 staff vs. no staff .68 .22 0.39“ .002 .67 .22 0.39" .002
3-5 staff vs. no staff .85 .24 0.43*** .000 .85 .24 0.43*** .000
6 0’ mm Staff“ ”0 .87 .28 0.31** .002 .91 .29 0.32** .002
staff
Survey Year 2006 vs.
2005 -.19 .12 -0.11 .107 -.19 .12 -0.12 .112
Training Breadth -.04 .03 -0. 15 .186
Training Frequency . 12 .06 0.18* .040
Traininglntensity .02 .09 0.02 .8 12
R2 0.16 0.18
F for change in R2 6.76*** .000 1.47 0.22
* P < .05; ** P < .01, *** P < .001
Table 21 . Training Dosge Predicting Comprehensiveness of Strategies
Model 1 Model 2
B SE B B p B S: [3 p

(Constant) -.42 .15 .004 -.65 .21 .002
Time 1
Comprehensive .23 .06 0.26*** .000 .23 .06 026*" .000
Strategies
Have DFC Gram VS' .62 .16 0.27*** .000 .65 .16 0.29*** .000
no Grant
Training Breadth -.01 .03 -0.05 .655
Training Frequency .03 .06 0.05 .607
Training Intensigy .12 .09 0.12 .180
R2 0.16 0.17
F for change in R2 16.77“"‘000 0.82 0.487

 

* P < .05; ** P < .01, *** P < .001

119

Table 22. Training Dosage Predicting Facilitation of Community Change

 

 

 

 

Model 1 Model 2
Variable B SBE B P 3 SB; ‘3 "

(Constant) -.56 .24 .021 —.60 .31 .056
1-2 staffvs. no staff .61 .26 0.31* .019 .63 .26 0.32* .018
3-5 staff vs. no staff .94 .28 0.42" .001 .98 .29 0.44" .001
:3me Staff VS' “0 .80 .32 025* .015 .82 .34 026* .014
Training Breadth -.03 .04 -0.08 .480
Training Frequency -.01 .07 -0.01 .880
Training Intensity .07 .10 0.07 .497
R2 0.06 0.07

F for change in R2 4.02" 0.008 0.35 0.787

 

* P < .05; ** P < .01, *** P < .001
Dosage Quartiles

To further explore the relationship between dosage and community capacity for
problem solving, the three dosage types were divided into quartiles and the bottom and
top quartiles were used as independent variables in the regression model. Other studies
have also divided participants into groups based on their level of attendance (e.g., high
vs. low participator) in order to compare outcomes among the attendance groupings
(Chaput et al., 2004). While this still allows a comparison of outcomes based on more or
less participation, it also informs potential cut-offs to compare outcomes determined by
levels of participation. Quartiles were used as cut-offs in this study since this can not be
informed by existing standards for training dosage cut-offs. However, since the previous
analyses did not indicate that there were incremental gains in community problem solving
capacity related to incremental increases in dosage, comparison of the top and bottom

quartiles offered another opportunity to examine the dose-response relationship for

120

coalition training. Nine separate regressions were run, testing each dosage quartile pair
to each DV. Covariates were entered into the first block and then the dummy coded
quartile dosage variable was entered into the second block, with the bottom quartile as
the reference group. This did reduce the sample size - Training Breadth Quartiles
analyses (n=l 19), Training Frequency Quartiles analyses (n=131) and Training Intensity
Quartiles analyses (n=137).

While the Multiple R for the regression models predicting each of the three
elements of community problem solving using only the covariates were statistically
significant, the addition of the dummy coded quartile dosage variables only significantly
increased the variance explained in one case (Tables 23-31). Specifically, the Multiple R
for Training Frequency Quartile predicting Coalition lntemal Functioning was
significant, F (4, 126) = 3.38, p<.001). R2 did increase significantly with the addition of
the training frequency quartile variable, (R2 Change=.03, p<.05). Examination of the
coefficients indicated that there was an effect for the top quartile of training frequency
([3: .17, t(125) = 1.99), p<.05), indicating that in comparison to the bottom quartile,
coalitions that were in the top quartile for training frequency experienced higher levels of
coalition internal functioning at post-test. Coalitions in the top quartile for training
frequency sent at least two or more individuals to additional CADCA training; while
coalitions in the bottom quartile did not send any individuals to additional training. Mean
scores for Time 2 lntemal Functioning for both groups were as follows: bottom quartile,

M=.02 (SD = .89) and top quartile, M=.35 (SD=.64).

121

Table 23. Training Breadth Quartiles Predicting Coalition lntemal Functioning

 

 

 

 

 

 

 

 

 

(n=1 19)
Model 1 Model 2
SE SE

Variable B B B p B B B p
(Constant) -.61 .29 .034 -.60 .30 .045
“me .1 18‘6”“ .14 .09 0.13 .128 .14 .09 0.13 .132
Functronrng
1-2 staffvs. no staff .58 .31 0.33 .060 .57 .31 0.33 .063
3-5 staffvs. no staff .98 .32 0.51" .002 .99 .33 0.51" .002
6 0' mm Staff“ 1.00 .37 0.36** .007 1.00 .37 0.36** .007
no staff
Top quartile vs. _ 02 16 -O 01 903
Bottom quartile ' ' ' ’
R2 0.13 0.13
F for change in R2 4.1 11”“ 0.004 0.02 0.902

* P < .05; ** P < .01, *** P < .001
Table 24. Training Frequency Quartiles Predicting Coalition lntemal Functioning
(n=1 3 1)
Model 1 Model 2

Variable B SE B [3 p B SE B [3 p
(Constant) .01 .25 .973 -. 15 .26 .560
“me .1 lf‘emal .20 .07 .23 .007 .19 .07 .22** .008
Functromng
1-2 staff vs. no staff .13 .25 .07 .611 .14 .25 .08 .559
3-5 staffvs. no staff .39 .27 .21 .145 .37 .26 .20 .165
6 0’ m0” smffvs' ”0 .45 .32 .16 .168 .50 .32 .18 .121
staff
Survey Year 2006 vs.
2005 -.20 .13 .13 .137 -.15 .14 -.09 .275
Top quartile vs. 27 14 17* 0 4 6
Bottom quartile ' ' ' '
R2 0.12 0.14
F for change in R2 3.30:”: .008 396* .049

 

* P < .05; ** P < .01, *** P < .001

122

Table 25. Training Intensity Quartiles Predicting Coalition lntemal Functioning
(n=137)

 

 

 

 

Model 1 Model 2

Variable B SE B B p B SE B 0 p
(Constant) -.44 .22 .051 -.32 .25 .21 1
“me .1 16‘6”" .20 .08 20* .015 .21 .08 22* .011
Functromng
1-2 staff vs. no staff .51 .24 .30“ .035 .48 .25 .28“ .049
3-5 staff vs. no staff .76 .26 .39** .004 .75 .26 .39“ .005
6 6’ “’6’6 “66"" “6 .78 .31 .29* .012 .74 .31 .27* .019
staff
T66 66‘" 616 VS' - 15 15 - 08 307
Bottom quartile ' ' ' °
R2 0.13 0.14
F for change in R2 4.98“: .001 1.02 .314

 

* P < .05; ** P < .01, *** P < .001

Table 26. Training Breadth Quartiles Predicting Comprehensiveness of Strategies
(n=119)

 

 

 

 

Model 1 Model 2
SE SE
Variable B B 6 p B B 6 p

(Constant) -.34 . 18 .055 -.34 .20 .080
Time 1
Comprehensive .34 .08 035*“ .000 .34 .08 0.35*** .000
Strategies
Have DFC Gm“ VS' .46 .20 020* .021 .46 .20 020* .022
no Grant
T66 quartile VS' 00 16 0 00 978
Bottom quartile ' ' ' '
R2 0.18 0.18
F for clmge in R2 13.10*** .000 0.00 0.96

 

* P < .05; ** P < .01, *** P < .001

123

Table 27. Training Frequency Quartiles Predicting Comprehensiveness of Strategies
(n=131)

 

 

 

 

Model 1 Model 2
SE
Variable B B 6 p 6 SE B 6 1’

(Constant) -.28 .16 .084 -.31 .17 .071
Time 1
Comprehensive .31 .08 0.33*** .000 .31 .08 033*" .000
Strategies
Have DFC Gram VS“ .46 .18 0.20* .013 .45 .19 0.20* .016
no Grant
Top quartile vs. 08 15 0 04 596
Bottom quartile ' ' ' '
R2 0.17 0.17
F for changg in R2 l3.34*** .000 0.29 0.60

 

* P < .05; ** P < .01, *** P < .001

Table 28. Training Intensity Quartiles Predicting Comprehensiveness of Strategies
fl=137)

 

 

 

Model 1 Model 2

Variable B SE B [3 p B SE B B p
(Constant) -.36 .16 .031 -.47 .20 .021
Time 1
Comprehensive .23 .07 0.25" .002 .22 .07 0.25” .002
Strategies
Have DFC Gram VS' .58 .18 0.26** .002 .60 .18 0.27** .001
no Grant
Top quartile vs.
Bottom quartile .15 .15 0.08 .337
R2 0.14 0.14
F for change in R2 10.77*** .000 0.94 0.334

 

* P < .05; ** P < .01, *** P < .001

124

Table 29. Training Breadth Quartiles Predicting Facilitation of Community Change

 

 

 

 

(n=1 19)
Model 1 Model 2
SE SE
Variable B B 6 p B B 6 1’
(Constant) -.25 .34 .462 -.17 .35 .619
1-2 staffvs. no staff .19 .36 0.10 .603 .17 .36 0.09 .638
3-5 staff VS. no staff .71 .37 0.33 .057 .73 .37. 0.34 .052
6 6’ more Staff“ .60 .43 0.19 .158 .57 .43 0.19 .180
no staff
Top quartile vs.
Bottomguartile -.15 .18 -0.08 .411
R2 0.07 0.08

 

* P < .05; ** P < .01, *** P < .001

Table 30. Training Frequency Quartiles Predicting Facilitation of Community Change

 

 

 

 

(n=1 3 1)
Model 1 Model 2
SE SE
Variable B B 6 P B B 6 p
(Constant) -.10 .28 .725 -.13 .29 .651
1-2 staff vs. no staff .07 .30 0.04 .808 .08 .30 0.04 .803
3-5 staff vs. no staff .42 .32 0.21 .181 .41 .32 0.20 .192
6 6’ "‘6’6 Staff“ .42 .38 0.13 .272 .43 .39 0.13 .265
no staff
Top quartile vs. 07 l 6 0 04 662
Bottom quartile ' ' ' '
R2 0.04 0.04

 

* P < .05; ** P < .01, *** P < .001

125

Table 31. Training Intensity Quartiles Predicting Facilitation of Community Change
(n=137)

 

 

 

 

Model 1 Model 2
SE SE
Variable 6 B 6 p B B 6 p
(Constant) -.36 .27 .173 -.37 .30 .218
1-2 staff vs. no staff .47 .29 0.25 .101 .48 .29 0.25 .103
3-5 staff vs. no staff .68 .31 032* .029 .68 .31 0.31* .030
6 6’ more “amt “6 .72 .36 023* .049 .72 .37 0.23 .051
staff
T6” quartile VS' 01 17 0 00 967
Bottom gartile ' ° ° °
R2 0.04 0.04

 

* P < .05; ** P < .01, *** P < .001
Proposition #1 .° Coalitions with both high training breadth and frequency will have the
highest levels of improvement in each of the three elements of community problem
solving capacity.

An analysis of a potential interaction between training breadth (i.e., the unique
number of individuals in a coalition trained) and training frequency (i.e., the total number
of individuals attending more than one training) was conducted to determine if higher
levels of community problem solving capacity were found when both a greater unique
numbers of individuals were trained and higher numbers of individuals received
additional training. Training breadth and frequency were both centered, and hierarchical
multiple regression was used, with the covariates entered into the first block, training
breadth and training frequency entered into the second block, and the interaction between
breadth and training entered into the third block (Tables 32-34).

For Coalition lntemal Functioning, the Multiple R for the regression model using

126

only the covariates was significant, but R2 did not significantly increase with the addition
of the dosage variables and interaction term. While examination of the coefficients does
indicate a main effect for Training Frequency, ([3: .18, t (174) = 2.07), p<.05), since the
overall model fit does not improve significantly, training frequency does not account for
a significant amount of overall variance in post-test Coalition lntemal Functioning. For
Comprehensiveness of Strategies and Facilitation of Community Change, while multiple
R for all the regression models using only the covariates was significant, there were no
main effects for either training breadth or training frequency, and the interaction between
training breadth and frequency was nonsignificant in the analyses. R2 did not
significantly increase with the addition of the dosage variables and interaction term.
Overall, the interaction of training breadth and frequency was not associated with any of

the elements of community problem solving capacity.

127

Table 32. Interaction of Training Breadth and Frequency Predicting Coalition Internal

 

 

 

 

Functioning

Model 1 Model 2 Model 3

SE SE SE

Variable B B B B B B B B B

(Constant) -.47 .22 -.46 .22 -.46 .22
g‘me.11?‘eml .22 .07 .23" .22 .07 .23** .22 .07 .23**
unctioning
l'zsmffvs'm .67 .22 .39** .66 .22 .39M .67 .22 .39"
staff
3‘58‘affvs'm .85 .24 43*" .84 .24 42*" .84 .24 42*M
staff
6°‘“‘°’es‘aff .87 .28 .31** .89 .28 .31M .89 .28 .31"
vs. no staff
SurveyYear .
200M810“ -.19 .12 -.11 -.19 .12 -.11 -.19 .12 -.11
Training Breadth -.O4 .03 -. 13 -.04 .03 -.13
“3mg .11 .06 ~.18* .12 .06 .18
Frequency
Training Breadth
XTraining .00 .01 -.01
Frequency
R2 0.16 0.18 0.18
szor change in 6.76"“ 2.19 0.02
R

 

* P < .05; ** P < .01, *** P < .001

128

Table 33. Interaction of Training Breadth and Frequency Predicting Comprehensiveness
of Strategies

 

 

 

 

Model 1 Model 2 Model 3
SE SE SE
Variable B B B B B B B B ‘3

(Constant) -.42 .15 -.41 .15 -.41 .15
Time 1
Comprehensive .23 .06 026*" .23 .06 026*“ .23 .06 026*"
Strategies
Have DFC Gram .62 .16 0.27*** .62 .16 0.27*** .62 .16 0.27***
vs. no Grant
Training Breadth .01 .03 0.04 .01 .03 0.05
gram” .01 .06 0.02 .02 .06 0.02

requency
Training Breadth
X Training .00 .01 —0.02
Frequency
R2 0.16 0.16 0.16
F f ha '
R2“ C ngc m 16.77*** 0.32 0.03

 

* P < .05; ** P < .01, *** P < .001

129

Table 34. Interaction of Training Breadth and Frequency Predicting Facilitation of
Community Change

 

 

 

 

Model 1 Model 2 Model 3
B SE B B B SE B B B SE B [3
Variable

(Constant) -.56 .24 -.55 .24 -.52 .24
1'2 Staff VS' “0 .61 .26 031* .60 .26 031* .60 .26 031*
staff
3'5 Staff VS' “0 .94 .28 0.42" .96 .28 0.43M .97 .28 0.43**
staff
6 or more staff vs.

.80 .33 025* .78 .33 024* .79 .33 0.25“
no staff
Training Breadth -.Ol .03 -0.03 .01 .03 0.04
Trammg -.02 .07 -0.03 .01 .07 0.01
Frequency
Training Breadth
X Training -.02 .01 -0. 18
Frequency
R2 0.06 0.07 0.09
F f ha ' 3.70
R2“ c "ge m 4.02** 0.30

 

*P<.05;**P<.

130

Proposition #2: The amount of training a coalition needs to build coalition capacity for
community problem solving will be related to the age of the coalition.

An exploratory analysis of a potential moderating effect for coalition age was
conducted to determine if the amount of training exposure needed to build coalition
capacity for community problem solving varied with coalition age. Nine separate
hierarchical regressions were conducted, with the covariates entered into the lSt Block,
the dosage measure and coalition age entered into the 2"d Block, and the interaction
between coalition age and the dosage variable entered into the 3rd Block. As in previous
analyses, while the multiple R for all analyses that included just the covariates was
statistically significant, the amount of variance explained did not increase significantly
when coalition age or the interaction between dosage and age were added. Coalition age
had neither a main effect on the dependent variables nor an interaction effect with any of
the dosage measures. Coalition age was not significantly associated with elements of
community problem solving; nor did the amount of training needed to build coalition

capacity for community problem solving vary with the age of the coalition (Tables 35-

43).

131

Table 35. Coalition Age as Moderator of Relationship between Training Breadth and

Coalition lntemal Functioning

 

 

 

 

Model 1 Model 2 Model 3

Variable B SEB p B SEB B B SE8 [3
(Constant) -.46 .21 -.39 .24 -.21‘ .27
Eumlllmma' .14 .07 015* .15 .07 016* .15 .07 017*
unctionmg
1'2 smffvs' “° .63 .22 0.39** .65 .22 0.40** .68 .22 0.42**
staff
3‘58‘affvs°“° .79 .24 0.43** .79 .24 0.43** .81 .24 0.44**
staff
“WWW” .84 .28 0.31** .85 .28 0.32** .86 .28 0.32**
vs. no staff
Training
Breadth .00 .02 0.01 -.05 .04 -0.17
Coalition Age -.01 .01 -0.07 -.03 .02 -0.22
Training
Breadth X .01 .00 0.25
Coalition Age
R2 . 0.10 0.11 0.12
Ff h '

0" angel“ 4.66** 0.50 1.88

R2

 

* P < .05; ** P < .01, *** P < .001

132

Table 36. Coalition Age as Moderator of Relationship between Training Frequency and

Coalition Internal Functioning

 

 

 

 

Model 1 Model 2 Model 3
Variable B SE B p B SE B p B SE B p
(Constant) -.37 .22 -.39 .25 .00 -.32 .26 .00
m“ .1 19‘3”“ .14 .07 .16 .15 .07 .16 .15 .07 .17*
Functlomng
1‘2 Sta“ VS“ "0 .61 .22 .38 .65 .22 .40 .65 .22 .40**
staff
3‘5 Staffvs' “0 .76 .24 .41 .75 .24 .41 .75 .24 .41**
staff
6 °’ more Staff .79 .28 .29 .84 .28 .31 .84 .28 .31**
vs. no staff
Survey Year -.16 .12 -.10 -. 13 .12 -.08 -.14 .12 -.09
“3mg .07 .04 .11 .00 . 10 .01
Frequency
Coalition Age -.01 .01 -.07 -.Ol .01 -. 12
Training
Frequency X .01 .01 .12
Coalition Age
32 0.11 0.13 0.13
F f h ‘
0' c angc m 4.12** 1.63 0.49

R2

 

* P < .05; ** P < .01, *** P < .001

133

Table 3 7. Coalition Age as Moderator of Relationship between Training Intensity and
Coalition Internal Functioning

 

 

 

 

Model 1 Mode12 Model 3
SE
Variable B SEB B B SE8 0 B B B
(Constant) -.46 .21 -.33 .28 -.13 .33
gum} 11mm“ .14 .07 015* .15 .07 016* .15 .07 017*
unctlonlng
1‘2 S‘affvs' “0 .63 .22 0.39** .64 .22 0.39** .66 .23 0.41**
staff
3'53mfivs'm .79 .24 0.43** .79 .24 0.43** .80 .24 0.43**
staff
6°’m°'esmff .84 .28 0.31** .84 .28 0.31** .84 .28 0.31**
vs. no staff
Training
Intensity -.02 .07 -002 -.13 .12 -0.15
Coalition Age -.01 .01 -0.08 -.03 .02 -0.24
Training
Intensity X .01 .01 0.22
Coalition Age
32 0.11 0.12 0.13
F for Change in 4.12** 0.59 1.42

R2

 

* P < .05; ** P < .01, *** P < .001

134

Table 38. Coalition Age as Moderator of Relationship between Training Breadth and
Comprehensiveness of Strategies

 

 

 

 

Model 1 Model 2 Model 3
SE SE
Variable B B B 3 SE B B B B '3

(Constant) -.25 .14 -.35 .20 -.24 .25
Time 1
Comprehensive .18 .06 0.21** .18 .06 0.22" .18 .06 0.22M
Strategies
Have DFC
Grant vs. no .47 .16 0.22" .47 .16 0.22" .47 .16 0.22"
Grant
Training
Breadth .01 .02 0.03 -.02 .04 -0.07
Coalition Age .00 .01 0.04 .00 .02 -0.04
Training
Breadth X .00 .00 0.14
Coalition Age
32 0.10 0.11 0.11
2;“ change in 9.60*** 0.23 0.61

 

* P < .05; ** P < .01, *** P < .001

135

Table 39. Coalition Age as Moderator of Relationship between Training Frequency and
Comprehensiveness of Strategies

 

 

 

 

Model 1 Model 2 Model 3
SE
Variable B SE B B B SE B B B B B

(Constant) -.25 .14 -.33 .19 -.32 .20
Time 1
Comprehensive . 18 .06 0.21** .18 .06 0.22" .18 .06 0.22"
Strategies
Have DF C
Grant vs. no .47 .16 0.22** .46 .16 0.22“ .46 . 16 0.22"
Grant
“8mg .02 .04 0.03 .00 .10 0.01
Frequency
Coalition Age .00 .01 0.04 .00 .01 0.03
Training
Frequency X .00 .01 0.03
Coalition Age
R2 0.10 0.11 0.11
F f han '

°' ° ge m 9.60*** 0.24 0.02

R2

 

* P < .05; ** P < .01, *** P < .001

136

Table 40. Coalition Age as Moderator of Relationship between Training Intensity and
Comprehensiveness of Strategies

 

 

 

 

Model 1 Model 2 Model 3
SE SE
Variable B B B B SE B B B B ‘3

(Constant) -.25 .14 -.48 .24 -.31 .31
Time 1
Comprehensive .18 .06 0.21** . 18 .06 0.22" .17 .06 0.21"
Strategies
Have DFC
Grant vs. no .47 .16 0.22" .50 .16 0.24” .49 .16 0.23"
Grant
Tram .07 .06 0.08 -.01 .12 -0.02
Intensuy
Coalition Age .01 .01 0.04 -.01 .02 -0.09
Training
Intensity X .01 .01 0.17
Coalition Age
R2 0.10 0.11 0.11
F f han '
R2“ C ge m 9.60*** 0.79 0.72

 

* P < .05; ** P < .01, *** P < .001

137

Table 41. Coalition Age as Moderator of Relationship between Training Breadth and

Facilitation of Community Change

 

 

 

 

Model 1 Model 2 Model 3
B SE B B B SE B B B SE B 13

Variable
(Constant) -.47 .25 -. 16 .29 .02 .32
1'2 3‘3““ “0 .56 .26 030* .59 .26 031* .62 .26 033*
staff
3'5 s‘affvs' ”0 .85 .28 0.40** .90 .28 042** .92 .28 0.43**
staff
6 0mm Staff .67 .33 022* .69 .33 022* .70 .33 022*
vs. no staff
Training
Breadth -.03 .02 -0.08 -.07 .05 -0.24
Coalition Age -.02 .01 -0.14 -.04 .02 -0.26*
Training
Breadth X .00 .00 0.21
Coalition Age
R2 0.05 0.08 0.09
F for chan e 'n
R2 g l 312* 2.27 1.35

 

*P<.05;**P<.01

138

Table 42. Coalition Age as Moderator of Relationship between Training Frequency and
Facilitation of Community Change

 

 

 

 

Model 1 Model 2 Model 3
B SEB [3 B SE3 0 B SEB 13

Variable
(Constant) -.47 .25 -.20 .28 -.03 .29
"zsmms'm .56 .26 030* .60 .26 032* .60 .26 032*
staff
3'58‘affvs'm .85 .28 0.40** .89 .28 0.42** .88 .28 0.41**
staff
6°rm°resmff .67 .33 022* .68 .33 022* .68 .33 022*
vs. no staff
gmmg -.06 .05 -0.08 -22 .12 032*
requency
Coalition Age -.02 .01 -0.14 -.04 .02 -0.25**
Training
Frequency X .02 .01 0.28
Coalition Age
32 0.05 0.08 0.10
Ff h
, 03° “g6 312* 2.18 2.59
lnR

 

*P<.05;**P<.01

139

Table 43. Coalition Age as Moderator of Relationship between Training Intensity and

Facilitation of Community Change

 

 

 

 

Model 1 Model 2 Model 3
B SE B [3 B SE B [3 B SE B [3
Variable
(Constant) -.47 .25 -.25 .33 .03 .39
1’2 Staff VS' .56 .26 030* .60 .27 032* .62 .27 033*
no staff
3'5 Staff VS' .85 .28 0.40** .87 .28 0.41** .88 .28 0.41**
no staff
6 or more
staffvs. no .67 .33 022* .70 .34 0.22“ .71 .34 023*
staff
”mg -.01 .08 -001 -.14 .15 -0.13
IntenSlty
Coalition Age -.02 .01 -0. 14 -.04 .03 -0.30
Training
Intensity X .01 .01 0.21
Coalition Age
R2 0.05 0.07 0.08
F for change 3,121“ 1.67 1.07

in R2

 

*P<.05;**P<.01

140

Further Examination of Other Factors Related to Community Problem Solving Capacity
Technical Assistance and its Relationship to Coalition Community Problem Solving
Additional analyses were conducted to determine if technical assistance predicted
coalition outcomes. The lnstitute’s previous evaluation suggested that technical
assistance (TA) was a key factor in coalition effectiveness (Foster-Fishman, Law & Ahn,
2008) so this finding was retested here. As part of the Annual Survey, coalitions reported
if they had received technical assistance in the form of email, phone or brief one-on-one
“personal coaching” in the last 12 months. Out of the total 269 sample, 40% of coalitions
indicated they had received TA in the last 12 months (n=108) and 60% indicated they
have not received TA (n=161). Results of t-test analyses comparing the no-TA and TA
groups are provided below in Table 44.

Table 44. Bivariate Relationships of Technical Assistance Status with Elements of
Community Problem Solving Capacity

 

Technical Assistance Status

 

Mean Comparisons M (SD) Sig.
No TA Received TA
(n=l6l) (n=108)
Time I
Coalition lntemal Functioning .005 .220 t = -1.76
(1.02) (0.92) (df= 267, p = .078)
.004 .108 = -.85
Comprehensiveness of Strategies (0.98) (0.98) (df = 267, p =.394)
Time 2
Coalition lntemal Functioning -. 130 .284 t = -3.43
(1.10) (0.74) (df= 267, p = .001)"
-.107 .212 t = -2.57
Comprehensiveness of Strategies (1.12) (0.77) (df = 267, p = .010) *
-.057 .131 t=-1.49
Facilitation of Community Changes (1.] 1) (0.84) (df = 267, p = .136)

 

*P<.01;**P<.001

The results indicated that the coalitions that had received Technical Assistance

did not vary from the non-Technical Assistance group at Time 1 in Coalition lntemal

Functioning or Comprehensiveness of Strategies. However, at Time 2, there were

141

significant differences between the two groups, with the TA group having higher
Coalition lntemal Functioning and Comprehensiveness of Strategies scores. There were
no group differences in Facilitation of Community Change.

The relationship of technical assistance to coalition capacity for community
problem solving was further examined with a series of hierarchical multiple regressions.
The technical assistance variable was dummy coded before it was entered into the
analysis as the independent variable, with the non-TA category as the reference group.
As in all previous analyses, Time 1 scores and covariates were entered into block 1 and
TA status was entered into block 2. Three separate regressions were run, one for each
dependent variable (Tables 45-47).

Multiple R for the regression model predicting Time 2 coalition internal
functioning using only the covariates was statistically significant (F (4, 264) = 10.87
p<.001), predicting 14% of the variance. Adding the technical assistance variable
significantly increased the amount of variance explained (17%) in post-test lntemal
Functioning (R2 Change=.02, p<.01). Technical assistance in the past 12 months was a
statistically significant predictor of Time 2 coalition internal functioning, (B = .16, t (263)
= 2.78, p < .01) with those coalitions who had received TA having higher scores in the
DV. However, adding the technical assistance variable to the Comprehensive Change
and Facilitation of Community Change models did not significantly increase the variance

accounted for in the dependent variables, nor was it a significant predictor of these DVs.

142

 

Table 45. Technical Assistance Predicting Coalition lntemal Functioning

 

 

 

 

 

 

 

 

 

 

Model 1 Model 2
SE SE
Variable B B B p B B B p
(Constant) -.39 . 18 .027 -.48 . 18 .006
gm .1 “item“ .29 .06 0.28*** .000 .27 .06 0.27*** .000
unctlonlng
1-2 staff vs. no staff .35 .19 0.18 .063 .33 .19 0.16 .083
3-5 staff vs. no staff .57 .21 0.24" .008 .52 .21 022* .015
6 0’ mm Staffvs' “0 .66 .25 0.20** .009 .612 .25 019* .014
staff
Received TA .32 .11 0.16" .005
R2 0.14 0.17
F for change in R2 10.87*** .000 7.70** 0.006
* P < .05; ** P < .01, *** P < .001
Table 46. Technical Assistance Predicting Comprehensiveness of Strategies
Model 1 Model 2
Variable B SE B [3 p B SE B [3 p
(Constant) -.48 . 12 .000 -.49 . 12 .000
Time 1
Comprehensive .23 .06 0.23*** .000 .23 .06 0.22*** .000
Strategies
Have DFC Gram VS“ .33 .14 015* .017 .32 .14 015* .022
no Grant
Budget $ 100,000-
$l99,999 vs. Less .44 .15 0.21" .003 .41 .15 0.19“ .007
than $100,000
Budget $200,000 and
above vs. Less than .38 .15 0.17* .012 .36 .15 0.16”“ .019
$100,000
Received TA .10 .12 0.05 .439
R2 0.15 0.16
F for change in R2 1 l.95*** .000 0.60 0.439

* P < .05; ** P < .01, *** P < .001

143

Table 4 7. Technical Assistance Predicting Facilitation of Community Change

 

 

 

 

Model 1 Model 2

Variable
(Constant) -.65 .19 .001 -.66 .19 .000
Budget $100,000-
$199,999 vs. Less .32 .14 015* .028 .30 .15 0.14 .050
than $100,000
Budget $200,000
and above vs. Less .20 .16 0.09 .210 .19 .16 0.08 .241
than $100,000
1-2 staff vs. no staff .46 .20 0.23“ .023 .46 .20 0.23* .024
3-5 staff vs. no staff .80 .23 0.33M .001 .79 .23 0.33" .001
6 0’ more Staffvs' .62 .27 018* .022 .62 .27 018* .023
no staff
Received TA .05 .13 0.02 .723
R2 0.08 0.08
F for change in R2 4.55** 0.001 0.13 0.720

 

* P < .05; ** P < .01, *** P < .001
Overall CADCA Touch

Since coalitions are able to receive both training and technical assistance from
CADCA’s Institute, further examination delved more closely into how the combination
of training and technical assistance would have differential impacts on coalition
outcomes. A variable was created categorizing the total sample (n=269) coalitions into 4
groups of varying levels/types of “CADCA Touch”: no training or technical assistance,
only training, only technical assistance, and both training and technical assistance. The

means and standard deviations are presented in Table 48.

144

Table 48. Means and SD for Community Problem Solving by CADCA Touch

 

 

 

 

 

 

N Mean SD

Time 1 lntemal no training or ta 65 -_15 1.17
Functioning , .

only training 96 .l l 0.90

only ta 22 .23 0.94

both training and ta 86 .22 0.91

Time 1 no training or ta 65 .00 095
Comprehensive , ,

Strategies only tralnlng 96 .00 1,01

only ta 22 .07 1.17

both training and ta 86 .12 094

Time 2 lntemal no training or ta 65 -_ 36 1,30
Functioning . .

only tralnlng 96 .02 0.91

only ta 22 .18 0.71

both training and ta 86 .31 0.75

Time 2 no training or ta 65 -29 131
Comprehensive . ,

Strategies only tralnlng 96 .019 095

only ta 22 .22 0.81

both training and ta 86 .21 0,76

Time 2 Facilitation no training or ta (,5 -, 34 1.13
of Community . .

Change only training 96 _13 106

only ta 22 .40 0.85

both training and ta 86 .06 0,83

 

A one-way analysis of variance was conducted to examine mean differences
among the four groups in the elements of community problem solving at pre-test (Table
49). The analysis yielded no significant differences among groups in Time 1 variables,
indicating there was no confound that coalitions with pre-existing higher levels of

functioning seeking out training and TA.

145

Table 49. ANOVA Table for CADCA Touch on Pre-Test Community Problem Solving

 

 

 

Sum of Mean
Squares df Square F Sig.
Between Groups 5.74 3 1.91 1.98 0.117
Pm" .1 11mm“ Within Groups 253.83 265 0.96
unctlomng
Total 259.56 268
Between Groups 0.88 3 0.29 0.30 0.827
mm 1 ‘. Within Groups 257.15 265 0.97
Comprehenslve
SWCE‘CS Total 258.03 268

 

To examine the effect for varying levels and types of CADCA touch, hierarchical
multiple regression was used to examine how well this variable predicted community
problem solving after Time 1 scores and covariates were controlled for. The CADCA
touch variable was dummy coded, with the no training/ta group used as the reference
category. The training combination variables were used to predict each of the dependent
variables. Covariates were entered in block 1 and the dummy coded CADCA touch
variables were entered into block 2.
lntemal Functioning

Multiple R for the regression model predicting Time 2 internal functioning using
only the covariates was statistically significant (F (4, 264) = 10.87, p<.001), predicting
14% of the variance. The addition of the CADCA touch dummy coded variables
significantly increased the amount of variance explained in post-test lntemal Functioning
(R2 Change=.04, p<.01). Examination of the coefficients indicated that there was an
effect for receiving Training Only ([32 .14, t(261) = 2.01), p<.05) and receiving Both
Training and TA ([3: .25, t(261) = 3.44, p<.01). In comparison to the no training/ta

group, receipt of training and training combined with TA were associated with higher

146

intemal functioning at time 2 (Table 50). Post-hoc analysis using the Bonferonni
procedure indicated that the Both Training/T A group had significantly higher levels of
post-test Coalition Internal Functioning than the No Touch (p < .001). The post-hoe
analysis did not find any differences among any of the other groups. It appears that the
highest post-test levels of lntemal Coalition Functioning are experienced by coalitions
that have received both training and TA. Figure 9 depicts the change in Time 1 to Time 2
scores for all four groups.

Figure 9. Time 2 Coalition lntemal Functioning by Survey Period and CADCA Touch

 

 

 

 

 

 

 

.5 0.4 1
g 0.3 7 h/.
0
a 0.2 T .— — - — — —‘ T -
=
g 0.1 _ . ........ —O - No Touch
5 0 A ----- .. - I - OnlyTralmng
‘3 0] . —nr- OnlyTA
'2' ' Os . +BothTraining/1A
E -0.2 7 ‘ x . .
. \ .
g '0.3 " ‘ \ . ..
U -0.4 r .

Time 1 Time 2

Data Collection Pen‘od

 

 

 

Paired t-tests were used to examine this finding further. Pre and post-test
differences in Coalition lntemal Functioning were conducted for the No Touch and the
Both Training/T A groups. For the No Touch Group, while they did experience a
decrease from Time 1 to Time 2, this change was not significant, t(64) = 1.21, p = . 228.
The Both Training/T A Group increased from Time 1 to Time 2, but this increase was not

statistically significant, t(85) = -.82 , p = .414.

147

Comprehensiveness of Strategies

While the overall model was significant (F (7, 261) = 6.93 p<.001) explaining
15% of the variance in Time 2 Comprehensiveness of Strategies, the addition of the
CADCA Touch variables did not improve model fit. Additionally, none of the
coefficients for training and TA were significant (Table 51).
Facilitation of Community Change

Multiple R for the regression model predicting Time 2 internal functioning using
only the covariates was statistically significant (F (5, 263) = 4.55, p<.01), predicting 8%
of the variance in Time 2 Facilitation of Community Change scores. The addition of the
CADCA Touch variables significantly improved model fit (R2 Change=.03, p<.05).
Examination of the coefficients indicated there was an effect for the Only Training group
([3= .17, t(260) = 2.23, p<.05) and Only TA group ([3= .17, t(260) = 2.54, p=.05). In
comparison to the no training/T A group, receiving either training or TA was associated
with higher Community Change scores at post-test (Table 52). Post-hoc analysis using
the Bonferonni procedure supported the findings from the multiple regression, indicating
that the Only Training group (p < .05) and the Only TA group (p < .05) had significantly
higher levels of Facilitation of Community Change than the No Touch. No other group

mean comparisons were statistically significant.

148

Table 50. CADCA Touch Predicting Coalition lntemal Functioning

 

 

 

 

 

 

 

 

Model 1 Model 2
SE SE
Variable B B B p B B ‘3 P
(Constant) -.39 . 18 .027 -.63 .19 .001
gm“ .1 19‘3”“ .29 .06 0.28*** .000 .26 .06 0.26*** .000
unctlomng
1-2 staffvs. no staff .35 .19 0.18 .063 .30 .19 0.15 .117
3-5 staff vs. no staff .57 .21 0.24“ .008 .49 .21 0.21“ .021
6 °’ “‘0“ Staff ”3' "0 .66 .25 0.20** .009 .61 .25 018* .016
staff
Only training vs. no *
training/ta .30 .15 0.14 .044
Only ta vs. no
training/ta .41 .23 0.11 .071
Training and TA vs. H
no training/TA .52 .15 0.25 .001
R2 0.14 0.18
F for change in R2 10.87*** .000 4.02““ 0.008
* P < .05; ** P < .01, *** P < .001
Table 51. CADCA Touch Predicting Comprehensiveness of Strategies
Model 1 Model 2
Variable B SE B [3 p B SE B [3 p
(Constant) -.48 .12 .000 -.54 . 13 .000
Time 1
Comprehensive .23 .06 023*" .000 .23 .06 022*" .000
Strategies
Have DFC Gram VS' .33 .14 015* .017 .32 .15 014* .029
no Grant
Budget $100,000-
$l99,999 vs. Less .44 .15 0.21" .003 .41 .15 0.20** .007
than $100,000
Budget $200,000 and
above vs. Less than .38 .15 0.17* .012 .37 .16 0.16* .017
$100,000
only "ammg VS' “0 .08 .16 0.04 .634

training/ ta

149

Table 51 (cont’d).

 

 

 

 

 

 

 

Only ta vs. no
training/ta .33 .24 0.09 .163
Training and TA vs.
no trainin g/T A .09 .18 0.04 .602
R2 0.15 0.16
F for change in R2 1 1.95*** .000 0.66 0.580
* P < .05; ** P < .01, *** P < .001
Table 52. CADCA Touch Predicting Facilitation of Community Change
Model 1 Mode12
SE SE
Variable B B B B B B B B
(Constant) -.65 . 19 .001 -.86 .20 .000
Budget $100,000-
$199,999 vs. Less .32 .14 0.15 .028 .26 .16 0.12 .097
than $100,000
Budget $200,000
and above vs. Less .20 .16 0.09 .210 .18 .17 0.08 .297
than $100,000
1-2 staff vs. no staff .46 .20 0.23* .023 .46 .20 0.22* .023
3-5 staff vs. no staff .80 .23 0.33" .001 .78 .23 0.32“ .001
B 0‘ mm“ Staff VS' .62 .27 018* .022 .63 .27 019* .020
no staff
Only training vs. no *
training/ta .36 .16 0.17 .026
Only ta vs. no *
training/ta .62 .24 0.17 .011
Training and TA vs.
no trainin s” A .19 .18 0.09 .298
R2 0.08 0.11
F for change in R2 4.55:.” 0.001 296* 0.033

 

* P < .05; ** P < .01, *** P < .001

Transfer of Learning

Additional analyses were conducted to determine the role transfer of learning

plays in predicting community problem solving capacity for the 182 coalitions that have

150

received training. A transfer of learning variable was created by averaging the responses
for 4 items in the 2008 Annual Survey capturing if in the past 12 months training,
technical assistance and publication materials from CADCA have been: 1) used to inform
decision making within the coalition; 2) shared with other individuals in the coalition; 3)
shared with others in the community (outside the coalition membership): and 4) if the
individuals trained/received technical assistance were still involved on the coalition. The
first three variables were on a 4 point scale ranging from 1 = not at all to 4 = a great deal.
The fourth variable was also on a 4-point scale, with 1 = none and 4 = all. The 4-item
scale had an alpha of .55 for the total sample (Dosage sub-sample a = .52, Untrained sub-
sample a = .62).

Mean value for transfer of Ieaming for the dosage sample was 3.30 (SD = .50).
Transfer of learning was not correlated significantly with any of the pre-test scores for
community problem solving. However, it was significantly positively correlated with all
the post-test scores for the DVs: lntemal Functioning, r(180) = .3 7, p < .001;
Comprehensiveness of Strategies, r(180) = .33, p < .01; Facilitation of Community
Changes, r(180) = .18,p < .05.

To examine this relationship further, hierarchical multiple regressions were used
to determine if transfer of learning significantly predicted Coalition lntemal Functioning,
Comprehensiveness of Strategies and Facilitation of Community Changes. Covariates
were entered into block 1 and the transfer variable was entered into block 2. Three
separate regressions were run, examining the relationship between transfer and each of

the three dependent variables.

151

lntemal Functioning

Multiple R for the regression model predicting Time 2 coalition internal
functioning using only the covariates was statistically significant (F (4, 177) = 7.57,
p<.001). The model predicted 15% of the variance in Time 2 coalition internal
functioning. When the transfer item was added, the model did improve significantly (R2
Change=.1 l, p<.001), explaining 26% of the total variance. Examination of the
coefficients indicated a significant main effect for transfer ([3 = .34, t (176) = 4.64, p <
.001), indicating that transfer of Ieaming is associated with higher levels of Coalition
lntemal Functioning (Table 53).
Comprehensiveness of Strategies

Multiple R for the regression model predicting Time 2 comprehensiveness of
strategies using only the covariates was statistically significant (F (2, 179) = 15.80,
p<.001). The model predicted 15% of the variance in Time 2 coalition internal
functioning. Transfer of Ieaming significantly improved the model fit (R2 Change=.07,
p<.01), with the model now explaining 22% of the total variance. Examination of the
coefficients indicated a significant main effect for transfer, ([3= .27, t(178) = 3.43, p<.01).
Greater comprehensiveness of strategies is associated with higher levels of transfer of
Ieaming (Table 54).
Facilitation of Community Changes

Multiple R for the regression model predicting Time 2 facilitation of community
change using only the covariates was statistically significant (F (3, 178) = 3.99, p<.01).

The model predicted 6% of the variance in Time 2 coalition internal functioning. Model

fit improved with the addition of the transfer variable (R2 Change=.03, p<.05), explaining

152

9% of the total variance in the DV. There was a significant main effect for transfer, ([3=

.16, t(177) = 2.15, p<.05), indicating greater transfer is associated with higher levels of

facilitation of community change (Table 55).

Table 53. Transfer of Learning Predicting Coalition lntemal Functioning

 

 

 

 

Model 1 Model 2

Variable B SE B [3 p B SE B [3 p
(Constant) -.59 .21 .004 -250 .45 .000
Time 1
lntemal .21 .07 0.22" .002 . 17 .06 0.18" .009
Functioning
"2 “w W ”0 .70 .22 0.41** .001 .71 .21 0.41** .001
staff
3‘5 Staff “‘3' “0 .89 .24 0.45*** .000 .84 .23 0.43*** .000
staff
6 0’ “‘0“ Staff .93 .28 0.33** .001 1.00 .26 035*** .000
vs. no staff
Transfer .58 .13 034*" .000
R2 0.15 0.26 .000
F f ha '
Rzor c nge in 757*" .000 27.13:“ .000

 

* P < .05; ** P < .01, *** P < .001

153

Table 54. Transfer of Learning Predicting Comprehensiveness of Strategies

 

 

 

 

 

 

 

 

 

Model 1 Model 2
SE SE

Variable B B B B B B B B
(Constant) -.42 . 15 .004 -l .85 .43 .000
Tmel
Comprehensive .23 .06 0.25*** .000 .23 .06 025*" .000
Strategies
”a“ DFC Gram .62 .16 0.27*** .000 .49 .16 021** .003
vs. no Grant
Transfer .47 .14 0.27** .001
R2 0.15 0.22
F f ha '
R2“ C nge m 15.80*** 0.000 8.10** 0.002

* P < .05; ** P < .01, *** P < .001
Table 55. Transfer of Learning Predicting Facilitation of Community Change
Model 1 Model 2
SE SE
B B p B B 9
Variable B B

(Constant) -.56 .24 .021 - l .59 .54 .003
1'2 Staff .61 .26 032* .019 .60 .26 031* .019
vs. no staff
3'5 “BB .94 .28 0.42** .001 .90 .28 0.41** .001
vs. no staff
6 or more
staffvs. no .80 .33 0.25* .015 .82 .32 026* .011
staff
Transfer .32 .15 016* .032
R2 0.06 0.09
F for
change in 3.99** 0.009 508* 0.047
R2

 

*P<.05;**P<.01

Transfer of Learning by Varying Levels of CADCA Touch

A one-way ANOVA was used to test if there were group differences among the

154

CADCA touch categories in amount of transfer of Ieaming. The total sample was used
and divided into the four CADCA Touch categories. The mean scores for the No Touch,
Training only, TA only and Both Training/T A groups for transfer of learning were 2.99
(SD = 0.68), 3.18 (SD = 0.53), 3.21 (SD = 0.71), and 3.43 (SD = 0.43). The ANOVA
revealed a significant effect for CADCA Touch, F (3, 265) = 7.74, p < .01). Post-hoc
analysis using the Bonferonni procedure indicated that the Both Training/T A group had
significantly higher levels of transfer than the No Touch (p < .01) and Only Training
Groups (p < .05). It appears that the highest levels of transfer are occurring with the
coalitions that have received both training and TA.
Testing CADCA ’s Theory of Change

CADCA’s theory of change was examined using hierarchical multiple
regressions. The theory of change posits that lntemal Functioning predicts
Comprehensiveness of Strategies which predicts Facilitation of Change. Multiple
regressions were performed to assess each of these relationships, with DF C status, budget
and staff covaried out. These analyses were run with the entire sample of coalitions
(n=269)
Relationship between Internal Functioning and Comprehensiveness of Strategies

A Coalition lntemal Functioning Change Score was created by subtracting Time 1
scores from Time 2 scores. The change variable was used as the predictor in the analysis.
The mean for the change score was -0.06 (SD = 1.14). Multiple R for the regression
model predicting Time 2 Comprehensives of Strategies using only the covariates was
statistically significant (F (3, 265) = 10.16, p<.001), predicting 10% of the variance in

Time 2 Comprehensiveness of Strategies. The addition of the lntemal Functioning

155

change score improved model fit (R2 Change=.08, p<.001). Examination of the
coefficient for the Internal Functioning Change variable indicated an effect ([3= .28,
t(264) # 4.86, p<.001), demonstrating that greater change between Time 1 and Time 2

lntemal Functioning scores was related to higher comprehensiveness of strategies (Table

 

 

 

 

56).
Table 56. Change in lntemal Functioning Predicting Comprehensiveness of Strategies
Model 1 Model 2
SE SE
Variable B B B p B B B p
(Constant) -.52 . 12 .000 -.49 .l 1 .000
Have DFC Gram VS' .40 .14 0.18“ .004 .35 .14 016* .010
no Grant
Budget $100,000-
$l99,999 vs. Less .41 .15 0.19” .006 .42 .14 0.20** .004
than $100,000
Budget $200,000 and
above vs. Less than .42 .16 0.18" .007 .48 .15 0.21“ .001
$100,000
Change ‘9 lntemal .25 .05 0.28*** .000
Functionifl
R2 0.10 0.18
F for change in R2 10.16*** .000 25.21*** .000

 

* P < .05; ** P < .01, *** P < .001

Relationship between Comprehensiveness of Strategies and Facilitation of Community

Change ‘
The change score for Comprehensiveness of Strategies was created by subtracting

Time 1 scores from Time 2 scores. The mean for the change score was -0.03 (SD =

1.21). Multiple R for the regression model predicting Time 2 Facilitation of Community

Change using only the covariates was statistically significant (F (5, 263) = 4.55, p<.01),

predicting 8% of the variance in Time 2 Comprehensiveness of Strategies. The addition

156

of the Comprehensiveness of Strategies change score improved model fit (R2
Change=.07, p<.001). Examination of the coefficient for the Comprehensiveness of
Strategies change score revealed an effect ([3= .27, t(262) = 4.68, p=.001), indicating that
higher scores differences in Time 1 and Time 2 comprehensiveness of strategies was
associated with greater facilitation of community change (Table 57).

Table 5 7. Change in Comprehensiveness of Strategies Facilitation of Community
Change

 

Model 1 Model 2

 

 

 

SE SE

Variable B B B p B B ‘3 p
(Constant) -.65 .19 .001 -.62 .18 .001
Budget $100,000-
3199,999 vs. Less .32 .14 015* .028 .17 .14 0.08 .221
than $100,000
Budget $200,000
and above vs. Less .20 .16 0.09 .210 .13 .16 0.05 .423
than $100,000
1-2 staff vs. no staff .46 .20 023* .023 .50 .20 025* .010
3-5 staffvs. no staff .80 .23 033** .001 .86 .22 036*** .000
B 0’ mm mm?" .62 .27 018* .022 .68 .26 0.20** .009
no staff
Change ‘B lntemal .23 .05 0.27*** .000
Functromng
R2 0.08 0.15
1: for changg in 32 4.55** .001 22.14*** .000

 

* P < .05; ** P < .01, *** P< .001

157

CHAPTER 5: DISCUSSION
This study sought to contribute to the existing research on how training can
improve anti-drug coalition capacity to be effective community change agents for
population-level reductions in substance abuse outcomes. This study examined whether a
coalition’s capacity for community problem solving increased with exposure to training.
By building a coalition’s community problem solving capacity, coalitions have the tools,
skills and knowledge to put into action key processes that can guide them toward distal
population-level health outcomes (Collie-Akers et al., 2007; F awcett et al., 2001; Lasker
& Weiss, 2003; Roussos & Fawcett, 2000). Furthermore, from a practical perspective,
training can target development of a coalition’s capacity to be an effective community
problem solver. The impact of training on three elements of coalition capacity for
community problem solving was examined in this study: coalition internal functioning,
comprehensiveness of strategies and facilitation of community changes. Training
provided by CADCA to coalitions targets improvements in each of these areas, and this
study focused on determining if improvements in community problem solving capacity
were related to greater exposure to training.
Key Findings
Overall, findings suggest that training, in general, is beneficial to coalitions.
Specifically, coalitions that have received training have higher levels of coalition internal
functioning, the first element of CADCA’s framework for community problem solving,
than those that have not received training. However, increased training participation does
not appear to lead to higher levels of community problem solving capacity. The dose-

response relationship for training and coalition outcomes was not strongly supported in

158

this study, and in general, coalitions did not accrue any additional benefit by exposure to
more training. Training’s overall link to improving coalition effectiveness in this model
appears to be complex and multi-faceted. Additional factors, including technical
assistance and transfer of Ieaming were examined. Coalitions that have received
. technical assistance in the last year also have higher levels of coalition internal
functioning when compared to those that have not received TA in the same time frame.
Furthermore, it appears that while coalitions benefit fi'om training and TA, they are most
helpful when implemented in an environment that supports the transfer of learning within
the coalition work context. Training, technical assistance and transfer of Ieaming all
appear to be important elements in understanding how supports to coalitions can be used
to improve their capacity in coalition internal functioning, comprehensiveness of
strategies and facilitation of community change.
Training Impacts on Coalition Capacity for Community Problem Solving

The first research question examined if training improved coalition capacity for
community problem solving using bivariate and multivariate analyses. First, independent
t-tests were conducted to examine potential mean differences in the dependent variables
between trained and untrained coalitions at pre-test. This was done to better describe the
sub-samples and rule out a potential confounding factor that coalitions with higher
capacities in community problem solving were also those that sought out training. There
was no such confound; at pre-test, both the trained and untrained coalitions were not
significantly different in Coalition lntemal Functioning and Comprehensiveness of

Strategies.

159

Hierarchical multiple regressions were conducted to determine if coalition
training status (trained and not trained) predicted each of the three post-test outcome
measures after controlling for pre-test levels and covariates of size, budget and DF C
funding. There were differences in Time 2 Coalition lntemal Functioning between the
trained versus untrained coalitions, with the trained group having higher scores. Even
after covarying out Time 1 Scores and coalition staff size, the differences were
significant, indicating that coalitions experienced benefits in their coalition internal
functioning (i.e., essential process usage, product use and quality, quality of membership
and climate for collaboration) when they received training from CADCA.

But how do these coalitions benefit from training? Initial analysis indicated that
the trained and untrained groups were similar in their pre-test levels of coalition internal
functioning at Time 1. However, at post-test, while those untrained were actually doing
significantly poorer in Coalition lntemal Functioning than those that had received
training, the trained coalitions did not experience improvements in their coalition internal
functioning from Time 1 to Time 2. Paired t-test analysis indicated that the trained
coalitions had the same Internal Functioning scores at pre and post test. This indicated
that the untrained group must have dropped in their Coalition lntemal Functioning from
Time 1 to Time 2. However, paired t-tests indicated that while the untrained coalition
group did have lower scores at Time 2, this drop was not statistically significant.

This finding is intriguing. While coalitions in this study came in with similar
levels of internal functioning, they may have had enough to start them on a road toward
being a better community problem solver. It may be that training acts as a potential

buffering agent in helping coalitions maintain their coalition internal functioning over

160

time. In other words, training may have an “inoculation” effect that protects coalitions
from other factors that may reduce their ability to be successful (e. g., changing funding
environment). This may allow coalitions enough time to work toward implementing
comprehensive strategies. Further follow-up is needed to determine if the decrease in
coalition internal functioning in the untrained groups continues overtime. If this is the
case, then the untrained group may be slowly spiraling downward to the point of
ineffectiveness.

Group differences between the trained and untrained coalitions in the covariates
of budget, DFC fimding status and staffing also indicate that in general, the untrained
coalitions have smaller budgets and fewer individuals working for the coalition, and few
have a Drug-Free Communities grant. This may result in fewer resources to invest into
training in comparison to their trained counterparts. Additionally, with less full-time
staff, these coalitions may not be able to spare anyone to send to training because of
person-power issues. The untrained coalitions also perceive themselves to be less mature
at post-test than the trained coalitions, giving further evidence that the lack of training
may hurt these coalitions over time. While trained and untrained coalitions have similar
levels of initial coalition internal functioning, funding and staffing factors may impede
untrained coalitions from accessing training, which may result in decreased internal
functioning over time.

Training status did not predict any of the other two elements of community
problem solving: comprehensiveness of strategies and facilitation of community change.
This examination suggests that training in and of itself is not sufficient to improve a

coalition’s use of comprehensiveness of strategies to address substance abuse or the

161

degree to which they help to bring about needed programs, policies and changes within
the community. While training is frequently used to support coalitions in their
development, (Florin, Mitchell & Stevenson, 1993; Quinby et al., 2008) and is considered
to be a key factor in their ability to achieve targeted goals and outcomes (Roussos &
Fawcett, 2000), the current study appears to suggest that training has very focused
impact. While coalitions benefited from training in this study, the benefit was
concentrated to coalition internal functioning. The implications for training suggest that
training should be applied strategically.

It should be noted that coalition internal functioning is the first step in CADCA’s
model of change. The training included in this study primarily emphasized improving a
coalition’s internal functioning, such as the development of quality planning products and
use of essential processes. The link between coalition training and coalition internal
functioning is consistent with other studies (Butterfoss, 2004; Butterfoss, Morrow,
Webster & Crews, 2003; Stevens & Lodl, 1999). Butterfoss’s (2004) study of coalition
training and TA for regional councils focused on improving perinatal health showed that
training and TA were successful at improving a coalition’s processes (e.g., coalition
decision-making, member recruitment) and structures (e.g., strategic plans). The
Communities That Care (CTC) study of training attendance found a positive relationship
between training attendance and coalition internal functioning, including clarity of
initiative goals, number of organizations working with the CTC initiative and stronger
board structure. However, there was no link between training attendance and perceived
efficacy of the CTC initiative. While the CTC logic model proposes that CTC training

impacts coalition functioning and this leads to higher project efficacy, that study was only

162

able to make the link of training to the first element of their logic model (Feinberg et al.,
2002). So this current study’s findings are consistent with other coalition training
studies. It may be that coalition training is best suited to building or maintaining a
coalition’s internal functioning, which is an earlier (if not the first) element of many
coalition building models (Butterfoss & Kegler, 2002 as cited in Butterfoss, 2007;
F einberg et al., 2002; Roussos & F awcett, 2000).

Dose-Response Relationship for Training and Community Problem Solving Capacity

The second and third research questions in this study delved more carefully into
how training involvement can impact coalitions, examining the degree of exposure
coalitions have had to training and how varying levels of exposure were associated with
varying coalition outcomes. Studies on prevention programs and out-of-school time
activities suggest that one reason why interventions have null findings is that degree of
exposure to the intervention has not been taken into account (August et al., 2004;
Charlebois et al., 2004; Rescinow et al., 2001). Training effects may be masked by not
taking into account how much training the coalition has received. This study applied
dose-response theory to coalition training to better understand this relationship.
Framing training from a dosage perspective has not been tested extensively in

prior research and so this study required an examination of testing various types of
training dosage and also determining the nature of the relationship between each training
dosage measure and coalition outcome (e.g., linear). For this study, three types of dosage
were examined: training breadth (number of unique individuals trained), training
frequency (number of individuals attending more than one training), and training

intensity (average number of coalition members attending each training).

163

One of the key foci of this study was to understand how the dose-response
relationship for coalition training and coalition outcomes should be defined. Research
Question 2 examined each training dosage’s unique relationship to each of the three
dependent variables. Scatterplots of each dosage measure and each Time 2 dependent
variable score were examined. Lines of best fit for linear, curvilinear and cubic trends
were examined but none were statistically significant. Examination of the lowess
smoothing line indicated that a linear relationship best fit the data. However, the slopes
were relatively flat for each dose/outcome plot, suggesting a weak relationship between
training dosage and each of the three elements of capacity for community problem
solving of interest in this study.

Other studies in other fields such as out-of-school time programs and
psychotherapy have the strongest evidence of a linear relationship between intervention
dosage and outcomes (Anderson-Butcher et al., 2003; August et al., 2004; Chaput et al.,
2004; Youniss, McLeIlan, Su & Yates, 1999). This study provided some weak support
for a linear relationship between coalition training dosage and outcomes. It may be that
the range restrictions in the data prohibited 3 firll examination of this relationship. Most
coalitions were in the lower levels of dosage in all three areas of breadth, frequency and
intensity and this may have reduced the ability to fully investigate the nature of the
relationship among the study variables.

While a threshold effect was not examined in this study because of the limited
range in the IVs, future studies of training dosage may benefit from this direction of
inquiry. While the out-of-school time studies offer some examples in which a threshold

model explained the dose-response relationship for program involvement and program

164

outcomes (Chaput et al., 2004), there is still much more work that needs to be conducted
in this arena. The time frame of training dosage follow-up studies may need to be
expanded to fully examine the specific nature of the dose-response relationship for each
dosage type and each outcome variable.

Research Question 3 focused on determining which type of training dosage
received was the strongest predictor of each element of community problem solving
capacity. Hierarchical multiple regressions were used and pre-test scores, staff, budget
and DF C funding status were covaried out. While the overall models were significant,
none of the dosage measures was a significant predictor of coalition outcomes. Further
examination of the dose response relationship for training investigated if outcomes varied
as a result of high or low dosage. Coalitions were grouped into quartiles for each dosage
type and then the highest and lowest quartiles were used to predict each of the three
elements of community problem solving. Here, there appears to be benefit at the highest
levels of training frequency - those coalitions who had two or more individuals attending
more than one CADCA training appeared to have higher levels of coalition internal
functioning than those coalitions who were not able to send any individuals to additional
training. This finding suggests that training frequency operationalized as the number of
individuals attending additional training may be a potentially useful way to examine
dosage. However, there may be a threshold effect in place, with a significant amount of
training frequency needed to benefit from additional exposure of this type. Again,
additional research is needed to further understand how training frequency may benefit a
coalition.

Two exploratory analyses were conducted to examine a potential interaction

165

effect for training breadth and frequency and for the moderating effect of coalition age on
the relationship between training dosage and outcomes. Proposition One that coalitions
would have stronger outcomes when there was both high training breadth and training
frequency was not substantiated. There was no significant interaction between training
breadth and frequency and having high levels of both dosage types did not benefit the
coalition in any of the outcome variables. While there is some limited evidence that
dosage is better conceptualized as a multi-faceted construct and the dose-response
relationship is strengthened by examining the interaction between dosage types (Welsh et
al., 2002), this finding was not supported by this study. However, given the weak
evidence of a dose-response relationship in this study, the lack of support for Proposition
One is not surprising. Future research should examine other types of dosage and their
interactions with each other to fully investigate a potential interaction between dosage
measures.

Proposition Two was also not substantiated; coalition age was not a moderator of
the relationship between training dosage and coalition capacity for community problem
solving. The amount of training a coalition receives does not appear to be a function of
how old the coalition is. In addition, coalition age by itself is not a significant predictor
of any of the elements of coalition capacity for community problem solving. This aligns
with the CADCA Institute’s evaluation findings that indicated the community problem
solving framework was consistent for both young and old coalitions (Foster-Fishman et
al., 2008). The CADCA Institute has never promoted a “stages of development” model
in their work with coalitions, and this finding gives support to their view that capacity for

community problem solving has nothing to do with how young or old a coalition is. This

166

implies that coalitions at any age may potentially benefit from becoming an effective
community problem solver.

In sum, there was some evidence of a relationship between training frequency and
coalition internal functioning, and coalitions seems to benefit from training but this
appears limited to only their internal functioning. Additionally, all of the regression
models predicting coalition outcomes with covariates were significant, but most
interestingly, Time 1 scores on lntemal Functioning and Comprehensiveness of
Strategies always accounted for the most variance in the model. These findings suggest
that where coalitions are at Time 1 is the biggest predictor of where they will be at Time
2.

It appears then, that the relationship between training and coalition outcomes is a
complex phenomenon, requiring additional scrutiny into other variables that may play a
role in improving a coalition’s capacity for community problem solving. For this
reason, additional analyses were conducted to examine the role technical assistance and
transfer of Ieaming played in building coalition capacity for community problem solving.

Technical Assistance and Community Problem Solving Capacity

Examination of the impact of technical assistance shows a similar story to that of
coalition training. Technical assistance has been found to be a critical factor in coalition
success (Florin et al., 1993). The Institute’s own external evaluation found that technical
assistance was an important trigger for coalition use of the community problem solving
model (Foster-Fishman etal., 2008). In this study, coalitions who reported receiving TA
in the last 12 months were compared to those who reported they had not received TA.

Those who had received TA had higher levels of coalition internal functioning at Time 2

167

than those who had not received TA. Results of regression analyses also indicated that
TA status was a significant predictor of Time 2 internal functioning, even after covariates
had been controlled for. These results suggest that technical assistance also plays a role
in helping a coalition develop its internal coalition capacity. However, similar to the
findings for training status, receiving TA in the last 12 months was not associated with
comprehensiveness of strategies or facilitation of community change. This corroborates
other training and TA studies that found TA beneficial to helping coalitions develop
aspects of their internal functioning (Butterfoss, 2004; Butterfoss et al., 2003). As with
training, CADCA’s technical assistance may best be suited to help a coalition build its
internal functioning.
CADCA Touch and Capacity for Community Problem Solving

Further analyses were conducted to examine if and how the combination of
training and technical assistance improves coalition capacity for community problem
solving. The dissection of the interplay between technical assistance and training
indicated that those coalitions who had been trained or trained and also received technical
assistance had higher levels of post-test internal functioning than those who had not
received any CADCA touch. In addition, the biggest mean score differences in Time 2
lntemal Functioning were between the no training/ta group and the group that had
received both training and TA. In this case, it appears that while receiving training and
technical assistance individually resulted in higher levels of internal functioning when
compared to receiving nothing at all, the best outcomes were experienced by coalitions
that had received both training and technical assistance.

As was found for the analyses examining the impact of training status, the No

168

Touch group did not experience significant decreases from pre- to post-test in Coalition
lntemal Functioning and the Both Training/T A group did not experience significant
increases fi'om Time 1 to Time 2. Again, this may mean that coalitions who have
received Training and TA have been adequately inoculated so that they are able to
maintain a level of coalition internal functioning that helps them keep up with work
sufficiently to the point in which they are able to implement a comprehensive set of
strategies. Follow-up studies are needed to determine if the pattern of change from pre to
post test experienced by the No Touch and Training/T A group continues to the point
where significant differences between Time 1 and Time 2 are observed. It may be that the
No Touch group experiences further decreases over time that makes these coalitions
unlikely to carry out strategies to effectively address their targeted substance abuse
problems while the Training/T A group grows in internal functioning over time to support
their ability to implement a comprehensive set of strategies targeting their local substance
abuse problems.

Varying levels of CADCA touch were not related to comprehensiveness of
strategies. Comparison of those that have not received any CADCA Touch, received
only training, received only TA, and received both training and TA indicated no
differences in these groups in post-test comprehensiveness of strategies.

Interestingly, higher levels of facilitation of community change were associated
with coalitions that had received only training or only technical assistance when
compared to coalitions that had not received any training or TA. There were no
differences between the coalitions that had received both training and TA with those that

had received nothing. Here, there appears to be no added benefit for receiving both

169

 

training and TA to improve a coalition’s involvement in community change. However, it
should be noted that since Time 1 scores were not measured for facilitating community
change, it may be, based on all the other analyses, that if Time 1 scores had been covaried
out, the effect for training and technical assistance would have been null.

It appears then, that the strongest link between CADCA touch and coalition
capacity for community problem solving is in coalition internal functioning.
Examination of the Institute’s theory of change depicted in Figure 1 may explain these
findings. The Institute’s theory of change for effective community problem solving
proposes a model by which strong internal functioning leads to coalitions pursuing a
comprehensive set of strategies and interventions, which leads to facilitating needed
community changes, which then contributes to the coalition’s ability to achieve
population level substance abuse decreases. Examination of the relationship between the
boxes in Figure 1 does indicate that facilitation of community changes is predicted by
greater change between Time 1 and Time 2 comprehensiveness of coalition strategies. In
addition, higher levels of comprehensives of strategies is associated with greater
increases between Time 1 and Time 2 coalition internal functioning. This study supports
the lnstitute’s theory of change.

Additionally, the strongest impact of CADCA “touch” appears to be in coalition
internal functioning, the first box in the theory of change used by CADCA. As discussed
earlier, here the coalitions who had received both training and technical assistance were
ones that had the highest levels of coalition internal functioning, when compared to the
no training/technical assistance group. While CADCA training and TA are designed to

impact each element of community problem solving, the training curriculum does

170

emphasize product development and use of the essential processes. Additionally,
CADCA believes these to be key precursor actions that need to be taken by a coalition in
order for it to become comprehensive, facilitate community change and contribute to
distal outcomes. This finding for internal functioning supports previous Institute
evaluation findings that suggested both training and TA trigger the process of change
(F oster-Fishman, Law & Ahn, 2008). In that study, training and technical assistance
were both needed to set the chain of events in motion. Additionally, training and TA had
a direct impact only on coalition internal functioning and not on the other boxes in the
model. This study corroborates that finding but also controls for where coalitions are at
pre-test, suggesting that while both training and TA by themselves are beneficial, the
strongest outcomes are experienced when training and TA are combined.
Implications for Coalition Training and Technical Assistance

One potential interpretation of these findings may be that the most appropriate
role for training and technical assistance is to trigger the theory of change process by
maintaining or building a strong internal capacity that will then help the coalition gain
momentum and move along the pathway, step by step, toward population level reductions
in targeted substance outcomes. The lack of a consistent relationship between training
and/or TA with other targeted outcomes of comprehensiveness of strategies and
facilitation of community change may indicate that it is not training or technical
assistance that improves capacity in these areas. The theory of change indicates that the
subsequent element of problem solving capacity is the key predictor of improvements. In
fact, the findings from this study suggest that training and technical assistance play a

critical role in developing capacity in the first element. It corroborates early CADCA

171

evaluation findings in that training and TA trigger the framework of change (F oster-
Fishman et al., 2008).

Additionally, transfer of learning appears to be a critical aspect of training use that
makes it more likely for coalitions to benefit from training. Training researchers have
posited that being trained, in and of itself, is not likely to bring about targeted
improvements (Baldwin & Ford, 1988; Rhodes et al., 2008). The training information
must be implemented and sustained within the organization setting to bring about
organization-wide changes. Successful transfer of Ieaming is a function of trainee-level
characteristics, how training itself is designed, and factors in the work environment itself
(Baldwin & Ford, 1988).

Examination of the transfer phenomenon was conducted in this study by
examining the work environment’s climate for transfer of learning, which included the
degree to while coalitions reported using and sharing training/T A information. Other
studies have found that organizations that developed a climate in their work environment
supporting transfer of skills and knowledge from training were more likely to see
implementation of training lessons in the organization (Burke & Baldwin, 1999; Saks &
Belcourt, 2006). The current study supported previous studies in that there were
significant associations for Ieaming transfer and coalition outcomes. Specifically, greater
internal functioning, comprehensiveness of strategies and facilitation of community
change among the 182 trained coalitions were associated with transfer of CADCA
training, technical assistance and publication information within the last 12 months. It
appears that the intake and use of CADCA materials is a critical factor in predicting

coalition outcomes. This was, by far, the only predictor that was able to account for a

172

significant amount of variance in all three of the dependent variables.

Additionally, the highest amounts of transfer occurred in coalitions that had
received both training and TA, suggesting that these might complement each other in
helping acquired skills and knowledge to be used in the coalition. This finding is
significant because while we know that transfer of training is not always successful
(Baldwin & Ford, 1988; Rhodes et al., 2008), it does appear to be occurring with the
coalitions in this study, especially those who have received both Training and TA.

One potential reason for the strength of this relationship may be because many
individuals trained by CADCA appear to be coalition leaders. While this information is
not tracked consistently in CADCA’s training attendance records, examination of results
from the evaluation of one of CADCA’s major training events, its 2009 Mid-Year
Training Institute, indicates that most attendees are coalition leaders. 37% of evaluation
respondents (n=3 94) indicated they were coalition directors. While the evaluation had a
response rate of 33%, this does give one indication that the individuals making the
decisions for the coalition are a large proportion of the training audience. Studies of
organization Ieaming climate indicate that supervisor and management support of
training and Ieaming transfer are key elements to building an environment supporting
Ieaming (McHargue, 2003; Saks & Belcourt, 2006). Coalitions in this study appear to be
very successful at the often elusive transfer of learning. This may be due to the buy-in to
training and implementation of CADCA’s theory of change within the coalition work
environment that results from coalition leaders being involved in training themselves.
They may be helping to foster the sharing of training information and building of systems

and processes so that training lessons are transferred and used in the day-to-day work of

173

coalitions.

It should be noted that training in this study was measured starting at baseline.
However, the TA measured here only reflects back on the past 12 months. Another
potential explanation for the transfer and CADCA touch findings may be that technical
assistance is the vehicle through which transfer occurs. This may suggest that technical
assistance can help with the transfer of knowledge and skills learned in training and
coalitions, regardless of when they have been trained, benefit from ongoing technical
assistance. While training may trigger the framework of change used by the CADCA
Institute, it is only successful within the context of transfer and transfer must happen
continuously through technical assistance.

In summary, the weak link between training dosage and coalition outcomes may
suggest that multiple exposures to training may not accrue any additional benefit for the
coalition or additional benefit from training only come at extremely highly levels of
exposure. In this study a single exposure appears to be sufficient to start the process of
change, especially if technical assistance is combined with training. It is possible that
CADCA training may be of sufficient quality and scope that attending a single training or
sending a single individual to training is enough to trigger the process toward being a
strong community problem solver.

However, the positive finding for the top quartile for training frequency requires
further investigation to examine the relationship between training dosage and coalition
outcomes. There is some suggestion with this finding that some benefits do accrue for
those coalitions that send individuals to additional trainings, but this benefit is only

experienced at the highest end of exposure. Given the time frame for this study, it may

174

P.

be that more time is needed to examine this phenomenon more accurately, especially
given range restrictions in the independent variables. Additionally, given the weak linear
relationship between training dosage and outcomes and nonsignificant results for a
quadratic or cubic relationship, there may be a threshold effect in place; however, the
range of training dosage in this study did not lend itself to examining this relationship
more fully.

In addition, it is possible that the types of dosage examined in this study may not
be the most relevant for training. The number of individuals trained, number of
individuals attending more than one training and average attendance at training provided
a first examination of the dose-response relationship for training and coalition outcomes.
There may be other dosage measures that may have more relevance. For example,
another conceptualization of training intensity may be the number of times more than one
person is sent to a training, rather than average attendance. This and other training
dosage measures should be examined in additional studies.

Another possible explanation for the lack of findings for dosage is that exposure
to other trainings, such as CTC training and training provided by state or regional-level
prevention providers, can not be controlled for. It is unrealistic to think that CADCA is
the only source of training accessed by these coalitions. Additionally, other entities
provide technical assistance services to coalitions. How all these support or contradict
the training and TA provided by CADCA is unknown. Regardless of the inability to
control for “Other Touch,” coalitions do appear to benefit from the training and technical
assistance they receive from CADCA. However, more research to delve systematically

into all the dosage of support received by a coalition is needed to better understand the

175

dose-response relationship for training and coalition effectiveness.

Finally, it seems that transfer of Ieaming is a critical element in predicting how
coalitions may benefit from training. The scope of this study did not include an
examination of what percent of individuals first trained were still with the coalition
presently. Additional investigation into longevity of trained staff/members may provide
further insight into how widespread training information is used and shared within the
coalition and with the larger community and how long the training effects last if there is
little turnover among staff that receive training.

Study Strength and Limitations

This study has several strengths. First, it offers an examination of a large and
diverse sample of substance abuse coalitions in this country. Rarely do coalition studies
have sample sizes as large as 269 groups. While it is impossible to determine the
representativeness of this non-random sample, the sample represents a wide range of
coalition budgets, age, staffing and geographic target areas.

Additionally, use of an extensive database of coalition training attendance records
offered an opportunity to examine multiple ways to conceptualize training dosage. Also,
CADCA’s Annual Survey of Coalitions offered a way to examine various elements of
coalition capacity for problem solving across hundreds of coalitions.

This study does have a few limitations that should be stated. Because secondary
data were used, the research questions were constrained by the data available. For
example, while examining training dosage based on training topic or type of individual
trained would be a valuable investigation, CADCA training records do not allow a fully

accurate assessment of this type of dosage. Additionally, while using training attendance

I76

records is suggested as a better way to calculate attendance dose than self-report of
receipt of training (McCall, Ryan, & Plemons, 2003), there may be error in these records,
with individuals either not signing in or signing in and leaving mid-training. No other
study of coalition training has compiled such an extensive training attendance history
(Feinberg et al., 2002; Quinby et al., 2008); however, this system is not without its faults.

Another limitation is that this study only covered a two-year period of time for
half the sample of coalitions, and there is the possibility that additional time is needed for
coalitions to fully benefit from training. For example, it may take more than two years
for a coalition to facilitate a policy change. While, other studies have examined coalition
outcomes in a similar timeframe and suggest that 48 months is sufficient time for
coalitions to bring about important improvements in the coalition and the community
(Guide to Community Preventive Services, 2001), more time may be needed to examine
this phenomenon fully. Given the ambitious nature of coalitions and the outcomes they
are targeting, changing long embedded community health conditions may take longer
than just a couple of years (Roussos & F awcett, 2002).

Conclusions and Recommendations for Future Research

The dosage measures examined in this study offered a first, comprehensive
examination of the dose-response relationship for training and coalition capacity for
community problem solving. For the most part, the dosage types selected for study here
do not seem to be related to community problem solving capacity; however, it does
appear that training and technical assistance in general are critical ingredients in fostering
strong coalition internal functioning that can then trigger a process by which coalitions

can become comprehensive in the strategies they implement, facilitate needed community

177

changed and contribute to population-level reductions in substance abuse problems.

Future studies should examine other types of training dosage and also attempt to
capture the dose-response relationship for technical assistance and coalition outcomes.
This study shows that training and technical assistance can impact coalition internal
functioning, and the best benefits may accrue when both are combined. Further inquiry is
needed to determine the best combination and timing of training, technical assistance and
other supports for coalitions.

In addition, dose quality may be another critical factor for training dosage that
should be examined. Bickman, Andrade and Lambert in their 2002 study of dose-
response for child and adolescent participants in mental health services suggest that one
reason they may not have found a dose-response relationship may be because they did not
measure the quality of the dose. When calculating the dosage score, the researchers
considered all mental health service contacts to be equal; however, a home visit is
qualitatively different and may carry a different “weighted” effect than a case
management contact. This potential measurement problem may be one reason why the
study failed to show an effect and points to the need to better understand what might be
more “important” aspects of participation than others.

Research on intervention implementation fidelity also suggests that it is important
to assess if an intervention has been implemented with the desired level of quality
(Charlebois et al., 2004; Dane & Schneider, 1998). Charlebois and colleagues (2004)
used observations and reports by trainers to assess if their program to reduce disruptive
behavior was implemented with integrity. Their study, while looking at attendance and

its relationship to program outcomes, also confirmed that the desired quality of the

178

program had been implemented, making a stronger case for the effectiveness of the
program. Future research should examine if training is implemented with quality,
including if trainers are providing training according to pre-defined guidelines and if
quality of the trainer has differential impacts on outcomes.

Further studies are required to determine how to best build coalition capacity to
be strong community problem solvers. While CADCA’s community problem solving
model appears to offer a strong theory of change for coalitions to achieve large-scale
decreases in substance abuse rates in their community, the process by which a community
problem solving framework is triggered requires additional investigation. While training
and technical assistance seem to play a role in launching a coalition in the right direction,
more research is needed to understand the best combination and timing of training,
technical assistance and other supports to achieve optimal results. Coalitions appear to be
on the right path if they move along the community problem solving road and future
studies will need to examine if the road leads to the ultimate destination of population-
level outcomes. What role training and dosage may play in this is not yet fully
understood and efforts need to be made to build a greater understanding of the process by
which community problem solving capacity is developed. The benefits of this
knowledge have broad impact: assisting the coalitions doing the work in their
communities, the intermediaries that provide training and supports to coalitions, and the
researchers who develop and study the best supports to help coalitions achieve their

goals.

I79

 

APPENDICES

180

APPENDIX A

Variable Comparison between 2005/2006 and 2008 Surveys

Table A. Variable Comparison for Time 1 and Time 2

 

COALITION INTERNAL FUNCTIONING

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Pre—Test Post-Test
2005/2006 2008
# of # of
Construct Survey Scale Survey Scale
Items Items
Item # Item #
Essential Processes
Assessin communi Average # 0f Avehafgc #
g ‘y #20 / #21 2 activities #34 7 . . .
needs and resources actrvrtres
used
used
Average #
. . . Average # of
“”1”sz ‘BfB’maBBB #20 / #21 1 activities #34 4 .°.f.
about the problem or goal actrvrtres
used
used
Average #
. Average # of
DBVBIBB'BB B BBB‘BWB’B #20 / #21 1 activities #34 4 .°.f.
of change actrvrtres
used
used
Average #
Developing and using #27-29 / 3 AZZiiafiii330f #34 3 of
strategic and action plans #28-30 activities
used
used
Identifying, adapting and Average # of Avegge #
implementing #20 / #21 2 activities #34 5 . . .
. . actrvrtres
mterventrons used
used
Average # of Avegige #
Evaluating the coalition #20 / #21 7 activities #34 12 . . .
actrvrties
used
used
Average #
. . . Average # of
BBSFB‘B‘BB B’BJBB‘S BBB #20 / #21 1 activities #34 7 .°.f .
initiatives actrvrtres
used
used
Planning Products
Develop/Revise Products Continuous
- Total number of Not used #36 (0_5)
products developed
Quality of Products - . Not used #38 Contmuous
Average product quality
Use Products - Average Not used #35 Continuous
product use
Membership Involvement
Quality of Membership — Continuous Continuous
Proportion of active #3 40 (0-40 #3 43 (0-43
coalition members sectors) sectors)
Collaborative Capacity Not used #9 5 Continuous

 

 

 

 

181

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Table A (cont’d).
COMPREHENSIVENESS OF STRATEGIES
Surve # of Surve # of
Construct Item z Items Scale Item i: Items Scale
Average involvement in Average of Average of
programmatic change #15 program #15 program
strategies items items
Average involvement in Average of Average of
environmental change # l 5 environment #1 5 environme
strategies al items ntal items
Total Program Strategy #15 Continuous #15 Continuous
(0-3) (0-3)
Total Environmental #1 5 Continuous #1 5 Continuous
Strategy (0-3) (0-4)
FACILITATION OF COMMUNITY CHANGES
Surve # of Surve # of
Construct Item z Items Scale Item #3; Items Scale
New/modified program #40 Categorical
chan e . (yes/no)
New/gmodified Not used #44 Categorical
policy/practice change (yes/no)
Expanded circle of #42 Continuous
influence with program (0-1 1)
changes Not used #46 Continuous
Expanded circle of (0-11)
influence with
policy/practice changes
Expanded targets of #43 Continuous
ro am chan e . (0-5)
Expiihded targets of N0, “58d #47 Continuous
Jolicy/practice changes (0-5)
Changes fit ““8810“ Of Not used #48-50 3 Continuous
coalrtron
Changes are strategic Not used #48-50 3 Copggqous

 

182

Tables B-C: Descriptive and Bivariate Statistics

Table B. Descriptive Statistics for Categorical Variables (Total Sample)

 

 

Variable Response 0/o (n)
Facilitation of Community Change
(Time 2 Only)
Program Change - Yes 71 (191)
Policy/Practice Change - Yes 60 (162)
Budget
Less than $100,000 38 (103)
$100,000 to $199,999 35 (94)
$200,000 and above 27 (72)
Number of Full-Time Equivalent
Staff
None 11 (29)
Fewer than 3 56 (150)
3-5 23 (63)
More than 6 10 (27)
DFC Grantee Status
Yes 71 (190)
No 29 (79)

 

183

Table C. Descriptive Statistics of Scales and Continuous Items (Total Sample)

 

TIME 1 TIME 2

 

M (SD) Range of Possible Possible

Scores

M (SD) Range of

Dimensions
Scores

Range

Range

 

COALITION INTERNAL F UNCTIONING

 

Essential Processes

 

Assessing
community needs
and resources
Analyzing
information about
the problem or goal
Developing a
framework of
change
Developing and
using strategic and
action plans
Identifying,
adapting and
implementing
interventions
Evaluating the
coalition
Sustaining projects
and initiatives

.82 (0.31)

.86 (0.34)

.83 (0.40)

.73 (0.42)

.71 (0.35)

.61 (0.35)

.80 (0.40)

0-1 0-1 .89 (0.20)

0-1 0-1 .90 (0.22)

0-1 0-1 .89 (0.27)

.92 (0.22)

0-1 0-1 .91 (0.22)

0—1 0-1 .83 (0.26)

0-1 0-1 .84 (0.25)

0-1

0-1

0-1

0-1

0-1

0-1

0-1

0-1

O-l

 

Planning Products

 

Develop/Revise
Products - Total
number of products
developed

Quality of Products
- Average product
quality

Use Products -
Average product
use

Not used

Not used

Not used

3.20(1.71)

3.20 (1.03)

3.64 (1.09)

0-5

04

0-5

04

 

Membership
Involvement

 

Quality of
Membership —
Proportion of active
coalition members

.61 (0.28)

0-1 O-l .82 (0.14)

.26-l

 

Collaborative
Capacity

 

Average
collaboration score

Not used

3.43 (0.45)

14

 

184

Table C (cont’d).

 

COMPREHENSIVENESS OF STRATEGIES

 

 

 

 

 

Average
mVBlVBmBB‘. ‘“ 3.28 (0.72) 1.334 0-3.67 3.45 (0.56) 1.334 14
programmatic
change strategies
Average
“‘VBIVB‘BBB‘ m 2.59 (1.15) 04 04.33 2.49 (0.73) 14 14
envrronmental
change strategies
Total Program
Strategy 2.94 (0.25) 0-3 1-3 2.90 (0.35) 1-3 0-3
Total
Environmental 2.68 (0.64) 0-3 0-3 3.20 (1.10) 0-4 04
Strategy
ACCOMPLISHING COMMUNITY/SYSTEMS CHANGES
Expanded circle of
influence with Not used 2.50 (2.43) 0-1 1 0-1 I
program changes
Expanded circle of
influence with
policy/practice N0! used '85 (233) 0-1 1 0-1 1
changes
Expanded targets
ofprogram change Not used 1.99 (1.69) 0-5 0-5
Expanded targets
of policy/practice Not used 1.66 (1.82) 0-5 05
changes
CBBBBBS. B‘ ““33”" Not used 2.91 (1.47) 04 04
of coalrtron
CBBBBBS are Not used 2.42 (1.18) 04 04
strategic
OTHER VARIABLES
11.02
Age (6.45) 0+ 0-37
(n=252)
Transfer Not used 3.22 (0.58) 1-4 1-4

 

185

 

 

Table D. Correlation Matrix of IVs, Transfer of Learning and DVs (Dosage Sub-
Sample)

 

 

1 2 3 4 5 6 7 8 9
1. Training Breadth --
2. Training Frequency .61* --
3. Training Intensity .61* .22* --

4. Coalition lntemal

Functioning Time I '08 '06 ~10 --

5. Comprehensiveness of

- * __
Strategies Time 1 '03 '01 -02 22

6. Coalition lntemal

- * an -_
Functioning Time 2 '03 '14 '03 ~28 -18

7. Comprehensiveness of * * *
Strategies Time 2 .06 .08 .05 .24 .29 .54 --

8. Facilitation of
Community Change -.02 -.03 .00 .18** .15" .44* .47* --
Time 2

** 4' 1! **
9_ Transferoueammg .08 .16 .03 .13 .04 .37 .33 .18 __

 

*P<.Ol;**P<.05.

186

Tables E-K. Results from Principal Components Analysis

Table E. Total Variance Explained for Time 1 Coalition lntemal Functioning Principal

 

 

 

Component Analysis
Initial Eigenvalues Extraction Sums of Squared Loadings
Component % of % of
Total Variance Cumulative % Total Variance Cumulative %

1 3.21 40.14 40.14 3.21 40.14 40.14
2 0.93 11.66 51.80

3 0.83 10.34 62.13

4 0.78 9.79 71.93

5 0.71 8.89 80.82

6 0.62 7.76 88.58

7 0.54 6.76 95.34

3 0.37 4.66 100.00

 

 

 

Table F. Total Variance Explained for Time 2 Coalition lntemal Functioning Principal
Component Analysis

 

Initial Eigenvalues

Extraction Sums of Squared Loadings

 

 

Component 0/0 of % of
Total Variance Cumulative % Total Variance Cumulative %

1 5.68 47.31 47.31 5.68 47.31 47.31
2 1.17 9.76 57.08 1.17 9.76 57.08
3 1.12 9.35 66.43 1.12 9.35 66.43
4 0.75 6.27 72.70

5 0.61 5.07 77.77

6 0.57 4.73 82.51

7 0.51 4.29 86.79

3 0.42 3.48 90.27

9 0.36 2.99 93.26

10 0.31 2.54 95.80

11 0.27 2.22 98.02

)2 0.24 1.98 100.00

 

187

 

Table G. Total Variance Explained for Time 1 Comprehensiveness of Strategies
Principal Component Analysis

 

Initial Eigenvalues

Extraction Sums of Squared Loadings

 

 

Component Total sziaiirfce Cumulative % Total vzfisrfce Cumulative %
1 2.49 62.21 62.21 2.49 62.21 62.21
2 0.86 21.52 83.73
3 0.44 10.97 94.70
4 0.21 5.30 100.00

 

Table H. Total Variance Explained for Time 2 Comprehensiveness of Strategies
Principal Component Analysis

 

Initial Eigenvalues

Extraction Sums of Squared Loadings

 

 

Component Total sziziifce Cumulative % Total Viriaiifce Cumulative %
1 2.49 62.29 62.29 2.49 62.29 62.29
2 0.92 23.09 85.37
3 0.39 9.79 95.16
4 0.19 4.84 100.00

 

Table 1. Total Variance Explained for Time 2 Facilitation of Community Change
Principal Component Analysis

 

Initial Eigenvalues

Extraction Sums of Squared Loadings

 

 

Component Total V22::ce Cumulative % Total sziriifce Cumulative %

1 4.50 56.26 56.26 4.50 56.26 56.26

2 1.61 20.10 76.36 1.61 20.10 76.36

3 0.96 12.01 88.36

4 0.28 3.55 91.91

5 0.25 3.14 95.05

6 0.21 2.67 97.72

7 0.15 1.82 99.54

3 0.04 0.46 100.00

 

188

Table J. Component Matrix for Time 2 Coalition lntemal Functioning Principal
Component Analysis

 

 

 

Component

1 2 3
Proportion of act1v1t1es coalrtrons were 1nvolved wrth 1n Assesses 0‘74 _021 -0.08
Community Needs section
Proportion of actrvrtres coalrtrons were 1nvolved wrth 1n Analyze 0.75 _0.14 0.06
problem" section
Proportion of actrvrtres coalrt1ons were involved wrth 1n Developed 0.73 -0.03 _0.01
Framewor " section
Proportion of activrtres coalrtrons were 1nvolved wrth 1n Develop 0.78 -0. 14 -0.06
Strategic Action Plan" section
Proportion of act1v1t1es coalrtrons were 1nvolved wrth 1n Identify 0.78 -0.15 0.03
adapt and implement intervention" section
Proportion of actrvrtres coalrtrons were 1nvolved wrth 1n Evaluate 0.82 _0.03 -0.01
Coalition" section
Proportion of actrvrtres coalrtrons were involved wrth 1n Sustain 0.74 0.02 0.04
projects and initiatives" section
Total number of products coalition developed or revrsed in last 12 0.57 0.07 -0.29
months
Average rating of how often coalition uses all 5 products 0-76 0'16 '0'06
Average rating of the degree to which each product 18 0.7] 0.16 -0.14
comprehensive and effective
Proportron of active coalition membership over total coalition 0.27 0.37 0. 59
membership
Rating of coalition climate for collaboration 0'35 0'42 0'51

 

189

 

Table K. Component Matrix for Time 2 Facilitation of Community Change Principal
Component Analysis

 

 

 

Component

1 2
Whether coalition brought about a new or modified policy or practice 0.67 0. 61
change
Whether coalition brought about a new or modified program 0-78 ’0-46
Total sectors where coalition brought about policy change 067 0-48
Total number of targets the coalition was trying to bring about a policy 0.67 0. 61
change in
Total sectors where coalition brought about program change 0-69 '0-45
Total number of targets the coalition was trying to bring about a program 0.76 -O.45
change in
Changes fit mission of coalition 0.87 -0.05
0-3 rating of whether of changes the coalition has brought about in last 12 0.8 6 -009

months have been strategic

 

I90

Figures A-I. Scatterplots of Training Dosage Types and Elements of Community Problem

Solving Capacity

Figure A. Scatterplot of Training Breadth and Coalition lntemal Functioning

Time 2 Internal Functioning

 

- 1.00000—

-2 .00000—

3.00000—

-4 .00000‘

0 O 000 I'IO.O(-Itii

0 (DUI

  

_____-‘

 

 

 

'5 3qu

0.00

 

 

I 1 l
10.00 15.00 20.00

Training Breadth

191

l
25.00

0 Observed
—Linear
—' Quadratic
— 'Cubic

Figure B. Scatterplot of Training Frequency and Coalition lntemal Functioning

 

 

 

Time 2 lntemal Functionina
1.11111 . 7 0 Observed
‘ 8 . ’ ° —Linear
.. -o— ._ '._.'.. '———=-' "' ~-~- —' Quadratic
0.""" 5 ' 8 " “Cubic
I a o 0
-1,11 1 1 1 0 o O
9
o
.1141 1 1
o
-3.1 1 1 1 1 '
O
-4_111 1 1
0.00 2.0.) 4.00 6.00 8.00
Training Frequency

 

192

Figure C. Scatterplot of Training Intensity and Coalition lntemal Functioning

Time 2 lntemal Functioning

 

 

 

 

 

 

00 o 0 9 x /
L ,__., "‘"
O
O
-3.W °
0
4W
0
1.00 2.00 3.00 4.00 5.00 600
Training Intensity

 

I93

0 Observed
—Linear
—' Quadratic
- 'Cuhic

Figure D. Scatterplot of Training Breadth and Comprehensiveness of Strategies

Time 2 Comprehensive Strategies

 

0 Observed
—Linear
, '—° Quadratic
" - -Cubic

 

 

 

 

 

I l I I I
0.00 5.00 10.00 15.00 20.00 25.“)

Training Breadth

 

I94

 

Figure E. Scatterplot of Training Frequency and Comprehensiveness of Strategies

Time 2 Comprehensive Strategies

 

0 Observed
—- Linear
""" Quadratic
- ' Cubic

 

 

 

 

1 1
0.00 2.00 4.00 6.00 8.00
Training Frequency

 

195

Figure F. Scatterplot of Training Intensity and Comprehensiveness of Strategies

Time 2 Comprehensive Strategies

 

0 Observed
—Linear
—° Quadratic
- ° Cubic

 

 

 

 

 

1 1
1.00 2.00 3.00 4.00 5.00 6.00
Training Intensity

 

196

Figure G. Scatterplot of Training Breadth and Facilitation of Community Change

Time 2 Facilitation of Community Change

 

3.000001 ° 0.53“"
— Linear
—' Quadratic
- ' Cubic

 

 

 

 

 

\
\
.\. \
‘fi.
\‘\.
\ ~1
q
0 O O O O O O \
-2.0W0 1 1 1 T 1
0.00 5.00 10.00 15.00 20.00 251!)

Training Breadth

 

197

Figure H. Scatterplot of Training Frequency and Facilitation of Community Change

Time 2 Facilitation of Communitv Change

 

 

 

 

 

0 Observed
'—I.inear
—° Quadratic
o - - Cubic
o
o
O
o
' “ab-0 0
0 8“ g\\
o \\
~\
\.
\.
b'\
o o
T I I
4.00 6.00 8.00
Training Frequency

 

198

Figure 1. Scatterplot of Training Intensity and Facilitation of Community Change

Time 2 Facilitation of Cormnunitv Change

 

 

 

 

 

3.00000“ 0 Observed
—Lurear
—' Quadratic
_. o 0 O — 'Cubic
2.00000 F o
0 o
1 0° 8 °
1.00000—> 09 o o 0 o /
103 g o ,
o 8 Cb 0 o o 0 o /
a (L ,L 0 /
0.00000— " U O‘O‘_fi__,,r —
1 o O 3%) O o o 0
4.00000“)
0
C00 000 0 o o o 0
200000 1 1 1 1 1
1.00 2 .00 3.00 4.00 5.00 6 .00

Training Intensity

 

199

APPENDIX B
2005 CADCA Annual Survey of Coalitions
2005 NATIONAL COALITION REGISTRY
Overview

The aim of CADCA’s National Coalition Registry is to identify coalitions around the
country and learn more about what they are doing in their communities to address
substance abuse problems. By adding your voice to the National Coalition Registry, you
will provide critically needed information that will advance the coalition field and help
inform CADCA of training and technical assistance needs that are vital to your coalition.
The National Coalition Registry will serve as a single source where coalitions,
policymakers, researchers and practitioners can access up-to-date information on
coalitions. Thank you for taking time to join the movement and making sure your
coalition is counted in the First Annual National Coalition Registry!

In addition to being included in the National Coalition Registry, completing the Registry
makes your coalition eligible to win the following prizes:

I Complete the registry by JANUARY 24, 2005 and be eligible to win a Computer
(1 Grand Prize)

I Complete the registry by JANUARY 31, 2005 and be eligible to win a Free
Weekend Stay at A Marriott Resort Property (1 Prize)

I Complete the registry by FEBRUARY 7, 2005 and be eligible for a Free
Registration to CADCA’s 2005 Mid-Year Training Institute (2 prizes) or
CADCA’s 2006 National Leadership Forum (2 prizes)

I All coalition that complete the registry by FEBRUARY 28, 2005 will be eligible
to win a $200.00 gift certificate from Office Depot (10 prizes) or a $200.00 gift
certificate from K-Mart (10 prizes)

Notice

The National Coalition Registry is administered and maintained by CADCA. The
collection of information for the National Coalition Registry is estimated to average 30
minutes per respondent, including the time for reviewing instructions, searching existing
data sources, completing and reviewing the information. Send comments regarding the
National Coalition Registry to CADCA, National Coalition Registry, 625 Slaters Lane,
Suite 300, Alexandria, VA 22314 or contact us at 1-800-54-CADCA or
registry@cadca.org.

General Instructions

The National Coalition Registry will take approximately 30 minutes to complete and asks

200

questions regarding your coalition’s characteristics, activities and focus. We are
requesting that the person in charge of the day-to-day operations of your coalition
(director, CEO, executive director, etc.) fill out the registry; however, we understand that
it may make sense for another person to fill out portions of the registry. In order to be
included in the Registry, please be sure to fully complete the ENTIRE registry by
FEBRUARY 28, 2005. Please mail back the Registry in the envelope that was provided
to you. The Registry packet must be postmarked no later than February 28, 2005 for your
coalition to be included in the Registry.

For your convenience, it may be helpful if you have the following information to refer to
as you fill out the Registry:

l) A copy of your strategic plan/annual report

2) Information on sector representation in your coalition

Disclosure Statement

Participation in the National Coalition Registry is voluntary. There is no penalty to you
or your coalition if you decide not to participate or choose to decline to answer any
particular questions in the Registry. The information requested in the Registry is
information about your coalition, and with the exception of some basic primary
respondent contact information, you will not be providing any personal information.
Information shared about your coalition in the National Coalition Registry will be
available to the public (including policymakers, funders, researchers, CADCA staff, and
other coalitions) via the CADCA website. However, individuals will have varying levels
of access to the information, according to the criteria listed below:

I Low level of access — This group of individuals includes the general public. They
will be able to browse the website and view general information about coalitions
via summary tables on information pertaining to coalition demographic
information; coalition background/description information; and coalition
strategies and interventions.

I Moderate level of access — This class of user includes all coalitions who
complete the Registry. Coalitions will need to use their unique user ID and
password (created during the Registry registration process) to log into a special
portion of the CADCA website to access information collected in the Registry.
They will be able to view all the information that the general public can view plus
information on training and technical assistance questions. These individuals will
also be able to produce summary reports on demand by selecting variables of
interest. They will be able to export data to Microsoft Excel for further
examination and analysis. Additionally, these individuals will be able to run a
summary report on their own coalition information so they can track progress
over time.

I Advanced level of access - The final class of user includes federal partners and
other government agencies as appropriate, researchers and CADCA staff. This
class of user will have full access to the web registry database (again with a
unique user ID and password). Although these individuals will not be able to

201

modify any elements of the database, their level of access will allow them to
cross-tabulate, filter and summarize any variable in the database. They will also
be able to export data for further examination and analysis.

Please check one of the following:

[:1 I have read the instructions, understand the purpose and use of the information,
agree to the above standards about the sharing of my coalition’s information with
the public, and give my consent to participate in the National Coalition Registry.
Please continue on to the next page.

B I decline to participate in the National Coalition Registry.

General Coalition Contact Information

Coalition Name:

 

 

 

 

DUNS #:
If your coalition has a DUNS # please provide that above.
Mailing Address:
City:
State:

 

Zip+4 (i. e., 22314-1212):

 

 

 

 

 

County:

Phone: ( ) Ext. _

Fax: ( ) Don’t have____
Email: Don’t have—
Website: Don’t have—

 

Is your street address the same as your mailing address?

Yes No

IF YES, PLEASE SKIP TO THE PRIMARY RESPONDENT SECTION
IF NO, PLEASE PROVIDE YOUR STREET ADDRESS BELOW

Street Address (if different from above):

 

202

City:

 

State:

 

Zip+4: (i.e., 22314-1212):

 

Please provide the nearest cross-streets to your coalition office/location:

 

Primagy Respondent Information

IF YOU ARE NOT THE DIRECTOR OF YOUR COALITION, PLEASE DIRECT
THIS REGISTRY T 0 THA T PERSON.

We are requesting that the person in charge of the day-to-day operations of your coalition
(director, CEO, executive director, etc.) fill out the Registry; however, we understand that
it may make sense for another person to fill out portions of the Registry. However, for

the purposes of this Registry, please provide contact information for the individual who is
the PRIMARY respondent.

Name (i.e., Ms. Joan Smith):

 

 

 

Phone: ( ) Ext.
Email: Don’t have
What is your title?

 

How long have you held this position?

Start Month Start Year

 

 

Are you responsible for the day-to-day operation and leadership of the coalition?
(i. e., you are the coalition leader, executive director, etc.)

Yes No

IF YES, PLEASE SKIP TO ITEM 1

If you are not the director, what is the name of the director of your coalition (if
applicable)?

 

203

 

 

1. Which of the following best describes the focus/activities of your coalition?
(check one only)

Mission focused solely on Alcohol, Tobacco and Other Drug issues (ATOD
specific)

ATOD issues as part of a broader mission/array of issues

__No focus on ATOD issues
(If you select this response, please stop. While we appreciate your interest in the
National Coalition Registry, the Registry is only for those coalitions with some
focus on ATOD issues. If you desire more information on community anti-drug
coalitions, please contact CADCA. Thank you for your time.) Please take a
moment to mail this Registry back to us in the stamped envelope that was
provided for you. By returning the Registry, you will still be eligible to win one
of the gift certificates to either Office Depot or Kmart. We thank you for your
time and appreciate your participation in the Registry.

 

 

 

2. Do any of the following 3. PLEASE ANSWER IF YOU CHECKED YES IN ITEM
organizations/groups help 2. How does this organization/group contribute to your
to plan and carry out your coalition’s efforts? (check all that apply)
coalition’s
strategies/efforts? (check
all that apply)

Y N Strategy/ Strategy/ Resources

E 0 Program Program (e.g.,

5 Development Implementation administrative,
operational, or
programmatic

supports; space;
funding; people)

 

Alcohol beverage control

 

Alcohol industry

 

Business groups

 

Child protective services

 

Citizen action groups

 

Civic/ fraternal
organizations

 

Courts

 

Elected officials

 

Employment services

 

Faith community

 

Government - county

 

Government - federal
(e.g., DEA, NHTSA)

 

 

 

 

 

 

Government - local

 

204

 

 

2. Do any of the following
organizations/groups help
to plan and carry out your
coalition’s
strategies/efforts? (check
all that apply)

3.

PLEASE ANSWER IF YOU CHECKED YES IN ITEM
2. How does this organization/group contribute to your
coalition’s efforts? (check all that apply)

 

Government — state

 

Government - tribal

 

Health

 

Housing authority

 

Human Services

 

Institutions of higher
Ieaming

 

Labor

 

Law enforcement

 

Local government _
executives

 

Mass media

 

Military Services (e.g.,
Air force, Army,
Marines, Navy)

 

National Guard

 

Parents

 

Prevention
providers/centers

 

Private Citizens (other
than parents and youth)

 

Probation

 

Public assistance

 

Recovery community

 

Recreation department

 

Elementary
School

 

Schools Junior High/
(K-l 2) Middle
School

 

 

High School

 

State ATOD agency
(SSA)

 

Transportation

 

Treatment
providers/centers

 

Volunteer organizations

 

Youth

 

 

 

 

 

 

205

 

 

 

2. Do any of the following 3. PLEASE ANSWER IF YOU CHECKED YES IN ITEM
organizations/groups help 2. How does this organization/group contribute to your
to plan and carry out your coalition’s efforts? (check all that apply)
coalition’s
strategies/efforts? (check
all that apply)

Other: I

Other: I

 

 

 

 

 

PLEASE NOTE: If less than 3 (THREE) organizations/groups help to plan and
carry out your strategies/efforts, we ask that you discontinue filling out the registry.
We thank you for your interest in the National Coalition Registry; however, we are
obtaining data on coalitions that have more sector involvement. If you would like
more information on community anti-drug coalitions and coalition development,
please contact CADCA. Thank you for your time.

Please take a moment to mail this Registry back to us in the stamped envelope that

was provided for you. By returning the Registry, you will still be eligible to win one

of the gift certificates to either Office Depot or Kmart. We thank you for your time
and appreciate your participation in the Registry.

Coalition Directopy Questions

4. Is YOUR COALITION the lead entity that sponsors coalition planning and/or
coordinates alcohol, tobacco and other drug-related programs, activities, policies

and strategies for or with other coalitions (i.e., coalition of coalitions)? PLEASE
NOTE — NOT TO BE CONFUSED WITH A FISCAL AGENT

Yes No

IF YES, PLEASE SKIP TO ITEM 6.
IF NO, PLEASE CONTINUE ON TO ITEM 5.

5. If you answered NO to QUESTION 4: Are you affiliated with a lead coalition?
Yes No

IF YES, PLEASE CONTINUE ON TO ITEM 6.
IF NO, PLEASE SKIP TO ITEM 7.

6. If you answered YES to QUESTION 4: What is the name of the lead coalition?

 

Please answer the following Questions from the perspective of the coalition you
represent:

206

l

\l

I If your coalition is the lead coalition (i.e., coalition of coalitions),
please answer from that perspective — DO NOT ANSWER ON
BEHALF OF THE COALITION S YOU LEAD.

0 If your coalition is an affiliate within a network of coalitions or a
freestanding coalition, please answer from that perspective.

7. When was your coalition first started?

Month Year

8. How does your coalition identify itself (i.e., which definitions best reflect your
coalition regarding scope and/or geography)? (check all that apply)

 

_City-wide
____Coalition of coalitions

_County/regional

_Faith-based

_Free-standing (i. e., concerned group of citizens and/or local agency
representatives)

_Neighborhood-wide

_School-based
___Statewide
_Other (specify)
_Other (specify)

 

 

9. Which category BEST describes the geographic area that you target (i. e., where
you focus your effort; population you want to impact)? (check one only)

___'City (please specify)
_Congressional District (please specify)
_County Qalease specify)
__Indian Reservation (please specify)
_Neighborhood/Community (please specify; provide geographic boundaries if
possible)
_Multiple Counties (please specify)
_Multiple Neighborhoods (please specify)
_School District (please specify)
_School Site
___Multiple School Sites (please specify)
_Other (please specify)

 

 

 

 

 

 

 

 

 

 

10. Do you target the entire geographic area or a particular sub-population in that
area (i. e., a smaller subset of the geographic area, e. g., at-risk youth,
Hispanic/Latino adults, teenage girls, etc. )?

207

___Entire Area

_Sub-Population Please identify, as specifically as possible, the
subpopulation(s) targeted by your coalition (e.g., American
Indian, Youth, Hispanic/Latino, etc.):

 

11. What was your COALITION ’s (not fiscal agent) annual budget for the last fiscal
year?

____Do not have a budget (IF CHECKED, PLEASE SKIP AHEAD TO ITEM 13.)

_$10,000 or less .. ‘
_$10,001 to $50,000 ‘
___$50,001 to $100,000
__$100,001 to $250,000
__$250,001 to $500,000
__$500,001 to $1,000,000 1
_$1,000,001 to $2,000,000 1
__$2,000,001 to $3,000,000

__$3,000,001 to $4,000,000 1
____$4,000,001 to $5,000,000 2
___$5,000,001 to $10,000,000

__$ 10,000,001 to $15,000,000

_$15,000,001 to $20,000,000

___More than $20,000,000

 

12. What percentage of your coalition’s total annual budget was devoted to: (please
estimate)

% Alcohol, tobacco, and other drug-related activities/ strategies
% Alcohol related activities/strategies only

13. How has your coalition’s budget changed over the past two years?
Increased
Decreased
Stayed the same/no significant change

14. How would you best describe your coalition? (check one only)
Novice

Intermediate
Advanced

Strategies and Interventions

Some coalitions provide direct services, some leverage/coordinate services, and some

208

do both. Please indicate your COALITION’S degree of involvement in each of the
following services/activities.

 

 

 

 

15. OPERATION 16. ACTIVITY LEVEL (check all that
(check all that apply) apply)
Coalition Coalition Plan Implement Coordina Provide
Staff Leverages/ te / Resourc
Provide Coordinates Monitor es (e.g.,
Direct Service or staff,
Service or Activity funding)
Activity
Community
policing programs

or services, for
example: ofiicers
who are out in the
neighborhood
working with
residents on crime
prevention

 

Information
dissemination,
such as:
brochures, fact
sheets, videos or
presentations

 

Community
mobilization, such
as: coalition
building or
neighborhood
watch

 

Community
development or
capacity building,
such as: training
and technical
assistance to
community
groups and
organizations

 

Development of
community laws
and policies that
discourage
substance abuse,
such as: zoning,
ordinances, server
training

 

Development of
community laws

and policies that

 

 

 

 

 

 

 

209

 

15. OPERATION

l6. ACTIVITY LEVEL (check all that

 

(check all that apply) apply)
Coalition Coalition Plan Implement Coordina Provide
Staff Leverages/ te / Resourc
Provide Coordinates Monitor es (e.g.,
Direct Service or staff;
Service or Activity funding)
Activity

 

SPECIFICALLY
discourage
ALCOHOL
abuse, such as:
zoning,
ordinances, server
training

 

Enforcement of
community laws
and policies that
discourage
substance abuse,
such as: citizen
watch groups

 

Enforcement of
community laws
and policies that
SPECIFICALLY
discourage
ALCOHOL
abuse, such as:
citizen watch
groups, server
stings

 

Media public
awareness, such
as: posters, public
service
announcements,
advertisements
and commercials

 

Media public
awareness
SPECIFIALLY
for ALCOHOL
problem
reduction, such
as: posters, public
service
announcements,
advertisements
and commercials

 

 

 

 

 

 

 

210

 

15. OPERATION

16. ACTIVITY LEVEL (check all that

 

(check all that apply) aPP'Y)
Coalition Coalition Plan Implement Coordina Provide
Staff Leverages/ te / Resourc
Provide Coordinates Monitor es (e.g.,
Direct Service or stafl,
Service or Activity funding)
Activity

 

 

Media advocacy,
media literacy and
social marketing
activities linked to
substance abuse
reduction

 

Media advocacy,
media literacy and
social marketing
activities linked
SPECIFICALLY
to ALCOHOL
problem reduction

 

Prenatal or
infancy programs
such as: maternal
and child health
care, nutrition or
child
development

 

Family support
programs such as:
family planning,
home visits from
health or social
service workers,
housing or child
care

 

Pre-marital
counseling

 

Peer leadership or
peer helper
programs

 

Life and social
skills training
programs such as:
assertiveness,
communication,
drug refusal,
problem-solving
or conflict
resolution skills

 

Teen drop-in
centers/clubs

 

 

 

 

 

 

 

211

 

15. OPERATION

l6. ACTIVITY LEVEL (check all that

 

(check all that apply) aI’DIY)
Coalition Coalition Plan Implement Coordina Provide
Staff Leverages/ te / Resourc
Provide Coordinates Monitor es (e.g.,
Direct Service or staff;
Service or Activity funding)
Activity

 

Youth support
groups, such as:
Alateen, COSA

 

 

Youth community
action groups,
such as: SADD,
youth councils

 

Drug—free social
and recreational
activities, such as:
drug-free dances,
‘Just Say No’
clubs, prom or
graduation
contracts

 

Brief
counseling/early
intervention
activities such as
substance abuse
assessment in the
emergency room,
primary care
offices, etc.

 

Increasing access
to treatment

 

Increasing
effectiveness of
substance abuse
treatment

 

Recovery support
activities such as:
sober housing,
peer support,
relapse prevention
or employment.

 

Re-entry support
activities for
offenders such as
returning from
incarceration

 

Other:

 

 

 

 

 

 

 

212

 

 

 

15. OPERATION l6. ACTIVITY LEVEL (check all that
(check all that apply) al’l’b’)
Coalition Coalition Plan Implement Coordina Provide
Staff Leverages/ te / Resourc
Provide Coordinates Monitor es (e.g.,
Direct Service or staff,
Service or Activity funding)
Activity
Other:

 

 

 

 

 

 

 

 

Monitoring, Evaluation and Training

17. Have you had any contact with CADCA/CADCA Institute staff in the last year?

Yes No

18. If you did, what type of contact did you have? (check all that apply)

CADCA Leadership Forum
CADCA Mid-Year Training Institute

Membership
Publications

Training/Technical Assistance

Website
Other (specify)

 

Other (specify)

 

19. Are you a current paid CADCA member?

Yes No

Please indicate which of the following activities your coalition has implemented
(either directly or in partnership) in the last 24 months.

 

20. Implemented
Activity
(check all
that apply)

21. Received information and/or
training on this topic from: (check
all that apply)

PLEASE NOTE: your coalition
may have engaged in a particular
activity, but may not have received

 

training.
TYPE OF ACTIVITY CADCA/CADCA Other Did not
Institute staff source receive
training

 

 

 

 

 

213

 

20. Implemented 21. Received information and/or

Activity training on this topic from: (check
(check all all that apply)
that apply)

PLEASE NOTE: your coalition
may have engaged in a particular
activity, but may not have received
training.

 

Collected data to determine or
monitor the extent of substance
abuse problems in the community
(Needs Assessment).

 

Collected data on risk and
protective factor predictors for
substance abuse and other related
problems.

 

Collected data on the readiness of
your community/system to select
and implement strategies found to
be effective in addressing
substance abuse.

 

Developed a logic model and
objectives for what the coalition
will accomplish.

 

Used data to prioritize substance
abuse needs.

 

Matched prioritized substance
abuse needs with evidence-based
programs/strategies.

 

Developed local or “homegrown”
programs/strategies to address
prioritized substance abuse needs.

 

Developed or selected
instruments and methods to
assess outcomes.

 

Collected data to monitor the
implementation of
programs/strategies.

 

Collected data to assess
immediate/intermediate outcomes
of programs/strategies.

 

Collected data to assess long-
term outcomes of
programs/strategies.

 

Reviewed

immediate/intermediate outcome

data to make decisions about
_flograms/strategies

 

 

 

 

 

214

 

20. Implemented 21. Received information and/or

Activity training on this topic from: (check
(check all all that apply)
that apply)

PLEASE NOTE: your coalition
may have engaged in a particular
activity, but may not have received
training.

 

Made changes to
programs/strategies based on
evaluation data.

 

Disseminated program evaluation
results and monitoring data to
key community stakeholders.

 

 

Re-designed evaluation methods
and/or measures to collect
better/additional information.

 

Used data as a basis for new
grants or funding proposals.

 

 

 

 

 

22. In general, how satisfied are you with the Technical Assistance and Training you
received from CADCA/CADCA Institute? (check one only)

_Very satisfied
_Somewhat satisfied
_Somewhat dissatisfied
____Very dissatisfied
___None received

23. To what extent has the Technical Assistance and Training you have received
from CADCA/CADCA Institute improved your capacity to provide effective
prevention services?

__A great deal
_Somewhat
_Not very much
_Not at all
_Not applicable

24. Does your coalition obtain training and technical assistance, materials, and/or
logistical support from any of the following? (check all that apply)

_Centers for Application of Prevention Technologies (CAPTs)
__Center for Substance Abuse Prevention (CSAP)

_Center for Substance Abuse Treatment (CSAT)

___Drug Enforcement Agency (DEA) Demand Reduction

215

gg

Pit
per

26

3‘:

.9

30

_National Clearinghouse for Alcohol & Drug Information (NCADI)
_National Guard

_National Highway Traffic Safety Administration (N HTSA)
_National Institute on Alcohol Abuse and Alcoholism (NIAAA)
_National Institute on Drug Abuse (NIDA)

Structure and Function

Please answer the following questions for your coalition only (i. e., not from the
perspective of a fiscal agent).

25.

26.

27.

28.

29.

30.

Does your coalition have a board or a governing body that sets the direction of
the coalition (i. e., a formal group or body in power that makes decisions for the
coalition)?

Yes No

Does your coalition have written bylaws/rules of operation?

Yes No
Does your coalition have a written strategic plan to reduce substance abuse in
your target area (i. e., a written document that specifies a coalition ’s aims, how it
will achieve its aims, and how it will be able to know if it was successful or not in
achieving those aims)?

Yes No

IF YES, PLEASE CONTINUE ON TO ITEM 28.
IF NO, PLEASE SKIP TO ITEM 30.

Does the written strategic plan have measurable outcomes?

Yes No

Has your coalition begun to implement its strategic plan?

Yes No
How many full-time equivalent paid STAFF work on alcohol, tobacco or other
drug activities, including in-kind staff (i. e., paid by someone else to work at your
coalition)? (check one only)

None 8 - 10
Fewer than 3 11 - 20
3 - 5 More than 20

216

6—7

31. How many VOLUNTEERS on average per month actively participate in your
coalition’s activities/strategies? (Volunteer: someone who does not get paid a salary
for their involvement in your coalition) (check one only)

None 1-5 6-10 1 1-25 26-50 More
than 50

Sustainability
32. Do you have a fiscal agent (i.e., an organization/entity that administers your funds)?

Yes No

 

IF YES, PLEASE CONTINUE ON TO ITEM 33.
IF NO, PLEASE SKIP TO ITEM 34.

33. Who is your fiscal agent?

 

34. Has your coalition been granted nonprofit status by the IRS (e.g., has 501(c)3
status)?

Yes No

IF YES, PLEASE CONTINUE ON TO ITEM 35.
IF NO, PLEASE SKIP TO ITEM 36.

35. What year was your coalition granted nonprofit status?

 

36. What sources of funding/resources support your coalition and its strategies?
(check all that apply)

____Asset forfeiture

__City government

_Corporate or Business

_County government

_Federal government — Direct from Agency
_Federal Government — Via Block Grant
_Federal Government — Other
_Foundations

_Fundraising

_In-kind contributions

_Non-profits (e.g., United Way)
____Private donations

___State government

217

Sales of products or Dues
Other (specify) ) ,

 

37. Please rank the top 5 largest sources of funding/resources (THAT YOU CHECKED
IN ITEM 36) that support your coalition and its strategies. (1 = highest level of

funding, 2 = next highest level of funding, etc.)

____Asset forfeiture

_City government

_Corporate or Business

_County government

_Federal government — Direct from Agency
_Federal Government — Via Block Grant
_Federal Government — Other
_Foundations

_Fundraising

_In-kind contributions

____Non-profits (e.g., United Way)
_Private donations

____State government

_Sales of products or Dues

Other (specify) J

 

38. Has your coalition received funding from any of the
following programs?
(check all that apply)

Currently
(2004-2005)

In the past
(Before 2004)

 

Drug Free Communities Support Program

 

Executive Office of Weed and Seed

 

Office of Juvenile Justice and Delinquency Prevention (OJJDP)
Underage Drinking Initiative

 

Safe and Drug Free Schools (local; state grant portion of the
20% set-aside; or national)

 

Substance Abuse Prevention and Treatment (SAPT) Block
Grant

 

State Incentive Grant (SIG) Sub-recipient

 

 

 

Selected Outcome Indicators

39. Does your coalition collect data on the following youth substance abuse outcome

indicators? (check all that apply)
Age of first use

Perception of disapproval by adults or peers
Perception of risk or harm

218

 

30-Day Use
Perceived availability

40. What instrument(s) do you use to collect your data? (check all that apply)

__ American Drug and Alcohol Survey (ADAS)

__ Communities That Care Survey (CTC)

__ Search lnstitute’s Developmental Assets Survey
__ Pride Survey (PRIDE)

__ Youth Risk Behavior Survey (YRBS)

__ Items/Scales from the CSAP Core Measures Initiative
_ Survey(s) sponsored/developed by your state.

Survey(s) that your coalition or community created/developed for your own
purposes

Other (specify)

 

Other (specify)

 

41. When did you start collecting survey data in your community?

Year

 

42. Who is responsible for coordinating/analyzing your survey data? (check all that
apply)

_City
_____Coalition
_College/University
_Consultant

_County

____Member organization staff person
_Research organization

_State
_Other (specify)
_Other (specify)

 

 

43. Does your coalition collect/utilize any of the following social indicator data?
(check all that apply)

_Alcohol consumption/sales data
_Alcohol/drug-related traffic fatalities data
_Alcohol outlet density data
_Arrest/incarceration data

_Census data

___Crime data

_Drug Abuse Warning Network (DAWN) data

219

_Emergency room data

_Health data

_National Registry of Substance Abuse Treatment Services (formerly the Uniform

Facility Data Set)
_National Registry on Drug Use & Health data (formerly the National Household
Registry on Drug Abuse)

_Other (specify)
_Other (specify)

 

 

 

220

 

44. Does your
coalition collect
data on any of

45. Based on available, verifiable data (e.g.,
Health Department data, substance abuse
incidence/prevalence data, Emergency

 

 

 

the following Room data, etc. ), which of the following
substances? are currently causing the most
problems in your community?
1 = the biggest problem
2 = the next biggest problem
3 = the third biggest problem
4 = fourth biggest problem
5 = fifth biggest problem
(check all that apply) RANK 5 ONLY
Alcohol
Amphetamines
Barbiruates

 

Cocaine (including Crack)

 

GHB

 

Hallucinogens (other than
LSD)

 

Heroin

 

Hydrocodone

 

lnhalants

 

LSD

 

Marijuana

 

MDMA (Ecstasy)

 

Methamphetamine
(including lce)

 

Nitrites

 

Oxycodone (Oxycontin)

 

PCP

 

Prescription Drugs (other
than Hydrocodone and
Oxycodone)

 

Rohypnol (date rape drug)

 

Steroids

 

Tobacco

 

Tranquilizers

 

Other:

 

 

Other:

 

 

 

 

221

APPENDIX C
2006 Annual Survey of Coalitions
2006 NATIONAL COALITION REGISTRY
Overview

The aim of CADCA’s National Coalition Registry is to identify coalitions around the
country and learn more about what they are doing in their communities to address
substance abuse problems. By adding your voice to the National Coalition Registry, you
will provide critically needed information that will advance the coalition field and help
inform CADCA of training and technical assistance needs that are vital to your coalition.
The National Coalition Registry will serve as a single source where coalitions,
policymakers, researchers and practitioners can access up-to-date information on
coalitions. Thank you for taking time to join the movement and making sure your
coalition is counted in the National Coalition Registry!

Different types of audiences will have different types of access to the information in the
Registry. Audiences with access include the general public, coalitions who complete the
Registry, and CADCA staff, researchers, federal partners and government agencies. The
general public will be able to view summary tables pertaining to coalition demographics,
descriptions, strategies and interventions. Coalitions who complete the Registry will
have a unique user ID and password so they can access information collected in the

- Registry. These coalitions will be able to view all the information that the general public
can view plus information on training and technical assistance questions. Finally, federal
partners and other government agencies as appropriate, researchers and CADCA staff
will have full access to the web registry database using a unique user ID and password.

General Instructions

The National Coalition Registry will take approximately 30 minutes to complete and asks
questions regarding your coalition’s characteristics, activities and focus. We are
requesting that the person in charge of the day-to-day operations of your coalition
(director, CEO, executive director, etc.) fill out the Registry; however, we understand that
it may make sense for another person to fill out portions of the registry.

For your convenience, it may be helpful if you have the following information to refer to
as you fill out the Registry:

3) A copy of your strategic plan/annual report
4) Information on sector representation in your coalition

Please read the following and put your initials in the box:
I have read the above information, understand the purpose and use of the

information, agree to the above standards about the sharing of my coalition’s
information with the public, and give my consent to participate in the National

 

 

 

 

222

Coalition Registry.

Notice - The National Coalition Registry is administered and maintained by CADCA.
The collection of information for the National Coalition Registry is estimated to average
30 minutes per respondent, including the time for reviewing instructions, searching
existing data sources, completing and reviewing the information. Send comments
regarding the National Coalition Registry to CADCA, National Coalition Registry, 625
Slaters Lane, Suite 300, Alexandria, VA 22314 or contact us at 1-800-54-CADCA or

registry@cadca.org.

Please check this box if your coalition is no longer active. You do not have to
complete the Registry but we ask that you contact CADCA at
registrygi‘cadcanrg or 800-542-2322, ext. 243 to let us know. If your coalition
should resume in 2006, please contact us so you can be counted in the 2006
Registry.

 

 

 

 

GENERAL COALITION CONTACT INFORMATION

Coalition Name:

 

 

 

 

DUNS #:
If your coalition has a DUNS # please provide that above.
Mailing Address:
City:
State:

 

Zip-+4 (i.e., 22314—1212):

 

 

 

 

 

 

County: Country:

Phone: ( ) Ext. __

Fax: ( ) Don’t have___
Email: Don’t have__
Website: I Don’t have____

 

Please provide the nearest cross-streets to your coalition office/location:

 

223

Is your street address the same as your mailing address?

Yes No

IF YES, PLEASE SKIP TO THE NEXT PAGE
IF NO, PLEASE PROVIDE YOUR STREET ADDRESS BELOW

Street Address (if different from above):

 

City:

 

State:

 

Zip+4: (i.e., 22314-1212):

 

************************************************************************

Please create a username and password that you will use to access the Registry’s
online reports. Be sure to record this information because you will need them to
access the Registry in the future.

USERNAME:
(Minimum of 8, maximum of 15 characters, no spaces or special characters such as *, @,
& etc.)

 

PASSWORD:
(Minimum of 8, maximum of 15 characters, no spaces or special characters such as *, @,
& etc.)

 

PRIMARY RESPONIENT INFORMATION

IF YOU ARE NOT THE DIRECTOR OF YOUR COALITION, PLEASE DIRECT
THIS REGISTRY T 0 THA T PERSON.

We are requesting that the person in charge of the day-to-day operations of your coalition
(director, CEO, executive director, etc.) fill out the Registry; however, we understand that
it may make sense for another person to fill out portions of the Registry. However, for

the purposes of this Registry, please provide contact information for the individual who is
the PRIMARY respondent.

Name (i.e., Ms. Joan Smith):

 

224

 

Phone: ( ) Ext.

 

Email: Don’t have

 

What is your title?

 

How long have you held this position?
Start Month Start Year

 

 

Are you responsible for the day-to—day operation and leadership of the coalition?
(i. e., you are the coalition leader, executive director, etc.)

Yes No

 

If you are not the director, what is the name of the director of your coalition (if
applicable)?

 

COALITION DIRECTORY QUESTIONS

 

2. Which of the following best describes the focus/activities of your coalition?
(check one only)

Mission focused solely on Alcohol, Tobacco and Other Drug issues (ATOD
specific)

ATOD issues as part of a broader mission/array of issues

No focus on ATOD issues

 

225

 

 

 

 

 

 

2. Do any of the following 3. PLEASE ANSWER IF YOU CHECKED YES IN ITEM 2.
organizations/groups How does this organization/group contribute to your
help to plan and carry coalition’s efforts? (check all that apply)
out your coalition’s
strategies/efforts?

(check all that apply)
Y N Strategy/ Strategy/ Program Resources
E 0 Program Implementation (e.g., space,
S Development funding, people,
administrative,
operational,
programmatic
supports)

Alcohol beverage

control

Alcohol industry

Business groups

Child protective

services

 

Citizen action groups

 

Civic/ fraternal
organizations

 

Courts

 

Elected officials

 

Employment services

 

Faith community

 

Government - county

 

Government - federal
(e. g., DEA, NHT SA)

 

Government - local

 

Government - state

 

Government - tribal

 

Health

 

Housing authority

 

Human Services

 

Institutions of higher
Ieaming

 

Labor

 

Law enforcement

 

Local government
executives

 

Mass media

 

 

 

 

 

 

226

 

2. Do any of the following 3. PLEASE ANSWER IF YOU CHECKED YES IN ITEM 2.
organizations/groups How does this organization/group contribute to your
help to plan and carry coalition’s efforts? (check all that apply)
out your coalition’s
strategies/efforts?
(check all that apply)

 

Military Services (e. g.,
Air force, Army,
Marines, Navy)

 

National Guard

 

Parents

 

Prevention
providers/centers

 

Private Citizens (other
than parents and
youth)

 

Probation

 

Public assistance

 

Recovery community

 

Recreation department

 

 

 

 

 

 

 

 

 

 

 

 

Elemen
tary
School
Junior
S h ls .
(23;) High/

Middle
School
High
School

State ATOD agency

(SSA)

Transportation

Treatment

providers/centers

Volunteer

organizations

Youth

Other:

Other:

 

 

 

 

 

 

 

4. Is YOUR COALITION the lead entity that sponsors coalition planning and/or
coordinates alcohol, tobacco and other drug-related programs, activities, policies

and strategies for or with other coalitions (i.e., coalition of coalitions, umbrella
coalition)? PLEASE NOTE — NOT TO BE CONFUSED WITH A FISCAL

227

AGENT

Yes No

IF YES, PLEASE SKIP TO ITEM 7.
IF NO, PLEASE CONTINUE ON TO ITEM 5.

5. If you answered NO to QUESTION 4: Are you affiliated with a lead coalition?

Yes No

IF YES, PLEASE CONTINUE ON TO ITEM 6.
IF NO, PLEASE SKIP TO ITEM 7.

6. If you answered YES to QUESTION 4: What is the name of the lead coalition?

 

Please answer the remainder of Qe Registgy from the perspective of the coalition
you represent:

0 If your coalition is the lead coalition or umbrella coalition (i.e.,
coalition of coalitions), please answer from that perspective - DO
NOT ANSWER ON BEHALF OF THE COALITIONS YOU LEAD.

0 If your coalition is an affiliate within a network of coalitions or a
freestanding coalition, please answer from that perspective.

7. When was your coalition first started?
Month Year Don’t Know

8. How does your coalition identify itself (i.e., which definitions best reflect your
coalition regarding scope) (check all that apply)

_Faith-based
__Free-standing (i. e., concerned group of citizens and/or local agency
representatives)

_School-based

_Serving an economically disadvantaged community

_Other (specify)
_Other (specify)

 

 

9. Which category BEST describes the geographic area that you target (i. e., where
you focus your eflort; population you want to impact)? (check one only)

228

_City (please specify)
_Congressional District (please specify)
_County (please specify)
_Indian Reservation (please specify)
_Neighborhood/Community (please specify; provide geographic boundaries if
possible)
_Multiple Counties (please specify)
_Multiple Neighborhoods (please specify)
_School District (please specify)
_School Site
_Multiple School Sites (please specify)
_Regional (please specify)
_Statewide
__Other (please specify)

 

 

 

 

 

 

 

 

 

 

 

10. Do you target the entire geographic area or a particular sub-population in that
area (i. e., a smaller subset of the geographic area, e. g., at-risk youth,
Hispanic/Latino adults, teenage girls, etc. )?

____Entire Area

_Sub-Population Please identify, as specifically as possible, the
subpopulation(s) targeted by your coalition (e.g., American
Indian, Youth, Hispanic/Latino, etc.):

 

11. What was your COALITION ’s (not fiscal agent) annual budget for the last fiscal
year?

Do not have a budget (IF CHECKED, PLEASE SKIP AHEAD TO ITEM 13.)

__$9,999 or less _$1,000,000 to $1,999,999
___$10,000 to $49,999 _____$2,000,000 to $2,999,999
__$50,000 to $99,999 __$3,000,000 to $3,999,999
_$100,000 to $199,999 __$4,000,000 to $4,999,999
_$200,000 to $299,999 ' _$5,000,000 to $9,999,999
___$300,000 to $399,999 __$10,000,000 to $14,999,999
___$400,000 to $499,999 _$15,000,000 to $19,999,999
_$500,000 to $999,999 _$20,000,000 and above

12. What percentage of your coalition’s total annual budget was devoted to: (please
estimate)

% Alcohol, tobacco, and other drug-related activities/strategies
% Alcohol related activities/strategies only

13. How has your coalition’s budget changed over the past two years?

229

Increased
Decreased
Stayed the same/no significant change

14. How would you best describe your coalition? (check one only)
Novice
Intermediate
Advanced
STRATEGIES AND INTERVENTIONS
Some coalitions provide activities directly, some leverage/coordinate activities, and

some do both. Please indicate your COALITION’S degree of involvement in each of
the following activities.

 

15. OPERATION
(check all that apply)

16. ACTIVITY LEVEL (check all that apply)

 

 

Coalition Coalition Plan Implement Coordina Provide
Staff Leverages/ te / Resources
Directly Coordinates Monitor (e.g., staj];
Provide Activity funding)
Acthity

Brief

counseling/early

intervention

activities such as
substance abuse
assessment in the
emergency room,
primary care
offices, etc.
Community
development or
capacity building,
such as: training
and technical
assistance to
community groups
and organizations
Community
mobilization, such
as: coalition
building or
neighborhood
watch
Community
policing programs
or services, for
example: officers
who are out in the

neighborhood

 

 

 

 

 

 

 

 

 

 

230

 

15. OPERATION

16. ACTIVITY LEVEL (check all that apply)

 

(check all that apply)
Coalition Coalition Plan Implement Coordina Provide
Staff Leverages/ te I Resources
Directly Coordinates Monitor (e.g., staf];
Provide Activity funding)
Activity

 

working with
residents on crime

prevention

 

Crime Prevention
Through
Environmental
Desig (CPTED)

 

Development of
community laws
and policies that
discourage
substance abuse,
e.g. zoning,
ordinances, server
training

 

Development of
community laws
and policies that
SPECIFICALLY
discourage
ALCOHOL abuse,
such as: zoning,
ordinances, server

tra g

 

Drug-free social
and recreational
activities, such as:
drug-free dances,
‘Just Say No’
clubs, prom or
graduation
contracts

 

Enforcement of
community laws
and policies that
discourage
substance abuse,
such as: citizen
watch groups

 

Enforcement of
community laws
and policies that
SPECIFICALLY
discourage
ALCOHOL abuse,
such as: citizen
watch groups,
server sti_ngs

 

 

 

 

 

 

 

231

 

15. OPERATION

l6. ACTIVITY LEVEL (check all that apply)

 

(check all that apply)
Coalition Coalition Plan Implement Coordina Provide
Staff Leverages/ te / Resources
Directly Coordinates Monitor (e.g., stajfi
Provide Activity funding)
Activi

 

Family support
programs such as:
family planning,
home visits from
health or social
service workers,
housing or child
care

 

Increasing access
to treatment

 

Increasing
effectiveness of
substance abuse
treatment

 

Information
dissemination, such
as: brochures, fact
sheets, videos or

presentations

 

Life and social
skills training
programs such as:
assertiveness,
communication,
drug refusal,
problem—solving or
conflict resolution
skills

 

Media advocacy,
media literacy and
social marketing
activities linked
SPECIFICALLY
to ALCOHOL
roblem reduction

 

Media advocacy,
media literacy and
social marketing
activities linked to
substance abuse
reduction

 

 

 

 

 

 

 

 

15. OPERATION

16. ACTIVITY LEVEL (check all that apply)

 

(check all that apply)
Coalition Coalition Plan Implement Coordina Provide
Staff Leverages/ te / Resources
Directly Coordinates Monitor (e.g., staff;
Provide Activity funding)
Activity

 

Media public
awareness
SPECIFIALLY for
ALCOHOL
problem reduction,
such as: posters,
public service
announcements,
advertisements and
commercials

 

Media public
awareness, such as:
posters, public
service
announcements,
advertisements and
commercials

 

Peer leadership or
peer helper

BIO g! ams

 

Recovery support
activities such as:
sober housing, peer
support, relapse
prevention or

emploment.

 

Re-entry support
activities for
offenders such as
returning from
incarceration

 

Teen drop-in
centers/clubs

 

Youth community
action groups, such
as: SADD, youth
councils

 

Youth support
groups, such as:
Alateen, COSA

 

Other:

 

Other:

 

 

 

 

 

 

 

233

MONITORING,L EVALUATION AN Q TRAIN INC

17. Have you had any contact with CADCA/National Coalition Institute staff in the
last year?

Yes No Don’t know
IF YES, PLEASE CONTINUE ON TO ITEM 18.
IF NO, PLEASE SKIP TO ITEM 20.

18. What type of contact did you have? (check all that apply)

_CADCA Leadership Forum

___Brief Technical Assistance (via phone or email)
_CADCA Mid-Year Training Institute
_____Training
_Membership
_Website
_Publications
__Other (specify)
___Other (specify)

 

 

19. In the last year, have you attended any of the following CADCA/National
Coalition Institute trainings?

__ 2-day training
__ 4-day training
_ Coalition Boot Camp Training
__ National Coalition Academy
__ Training of Trainers
__ Other (please specify)

 

Other (please specify)

 

Did not attend a CADCA/National Coalition Institute Training

20. Are you a current paid CADCA member?

Yes No

Please indicate which of the following activities your coalition has implemented
(either directly or in partnership) in the last 24 months.

234

 

21. Implemented
Activity

(check all that
apply)

22. Received information and/or
training on this topic from:
(check all that apply)

PLEASE NOTE: your
coalition may have engaged in
a particular activity, but may
not have received training.

 

TYPE OF ACTIVITY

CADCA/ Other Did not

National source receive
Coalition training
Institute

staff

 

Collected data to determine or monitor
the extent of substance abuse problems
in the community (Needs Assessment).

 

Collected data on risk and protective
factor predictors for substance abuse
and other related problems.

 

Collected data on the readiness of your
community/system to select and
implement strategies found to be

effective in addressing substance abuse.

 

Developed a logic model and
objectives for what the coalition will
accomplish.

 

Used data to prioritize substance abuse
needs.

 

Matched prioritized substance abuse
needs with evidence-based
programs/strategies.

 

Developed local or “homegrown”
programs/strategies to address
prioritized substance abuse needs.

 

Developed or selected instruments and
methods to assess outcomes.

 

Collected data to monitor the
implementation of programs/strategies.

 

Collected data to assess
immediate/ intermediate outcomes of
programs/strategies.

 

Collected data to assess long—term
outcomes of programs/strategies.

 

Reviewed immediate/ intermediate
outcome data to make decisions about
programs/strategies

 

Made changes to programs/strategies
based on evaluation data.

 

 

 

 

 

235

.__..__..——-_ ..

 

21. Implemented
Activity

(check all that
apply)

22. Received information and/or
training on this topic from:
(check all that apply)

PLEASE NOTE: your
coalition may have engaged in
a particular activity, but may
not have received training.

 

Disseminated program evaluation
results and monitoring data to key
community stakeholders.

 

Re-designed evaluation methods and/or
measures to collect better/additional
information.

 

Used data as a basis for new grants or
funding proposals.

 

NONE

 

 

 

 

 

23. In general, how satisfied are you with the Technical Assistance and Training you
received from CADCA/CADCA Institute? (check one only)

_Very satisfied
_Somewhat satisfied
_Somewhat dissatisfied
_Very dissatisfied
_None received

24. To what extent has the Technical Assistance and Training you have received
from CADCA/National Coalition Institute improved your capacity to provide
effective prevention services? (check one only)

____A great deal
_Somewhat
__Not very much
__Not at all
_Not applicable

25. Does your coalition obtain training and technical assistance, materials, and/or
logistical support from any of the following? (check all that apply)

Centers for Application of Prevention Technologies (CAPTs)

Center for Substance Abuse Prevention (CSAP)
Center for Substance Abuse Treatment (CSAT)

Drug Enforcement Agency (DEA) Demand Reduction
National Clearinghouse for Alcohol & Drug Information (NCADI)

National Guard

236

____National Highway Traffic Safety Administration (NHTSA)
_National Institute on Alcohol Abuse and Alcoholism (NIAAA)
_National Institute on Drug Abuse (NIDA)

_NONE

_STRUCTURE AND FUNCTION

Please answer the following questions for your coalition only (i. e., not flom the
perspective of a fiscal agent).

26. Does your coalition have a board or a governing body that sets the direction of
the coalition (i. e., a formal group or body in power that makes decisions for the
coalition)?

Yes No

27. Does your coalition have written bylaws/rules of operation?

Yes No

28. Does your coalition have a written strategic plan to reduce substance abuse in
your target area (i. e., a written document that specifies a coalition ’s aims, how it
will achieve its aims, and how it will be able to know if it was successful or not in
achieving those aims)?

Yes No

IF YES, PLEASE CONTINUE ON TO ITEM 29.
IF NO, PLEASE SKIP TO ITEM 31.

29. Does the written strategic plan have measurable outcomes?

Yes No

30. Has your coalition begun to implement its strategic plan?

Yes No

31. How many full-time equivalent paid STAFF work on alcohol, tobacco or other
drug activities, including in-kind staff (i. e., paid by someone else to work at your
coalition)? (check one only)

None Fewerthan3 3—5 6—7 8—10 11
— 20 More than 20

32. How many VOLUNTEERS on average per month actively participate in your

237

 

 

coalition’s activities/strategies? (Volunteer: someone who does not get paid a salary
for their involvement in your coalition) (check one only)

None 15 6-10 1 1—25 ' 26-50 More
than 50

SUSTAINABILITY
33. Do you have a fiscal agent (i. e., an organization/entity that administers your funds)?
Yes No

IF YES, PLEASE CONTINUE ON TO ITEM 34.
IF NO, PLEASE SKIP TO ITEM 35.

34. Who is your fiscal agent?

 

35. Has your coalition been granted nonprofit status by the IRS (e. g., has 501(c)3
status)?

Yes No

IF YES, PLEASE CONTINUE ON TO ITEM 36.
IF NO, PLEASE SKIP TO ITEM 37.

36. What year was your coalition granted nonprofit status?

 

 

 

Don’t know
37. Has your coalition received funding from any of the following Currentl In the past
programs? (20054008 (Before
(check all that apply) 2005)

 

Drug Free Communities Support Program

 

Drug Free Communities Support Program Mentoring Grant

 

Community Capacity and Development Office (formerly the Executive
Office of Weed and Seed)

Office of Juvenile Justice and Delinquency Prevention (OJJDP) Underage
Drinking Initiative

 

 

Safe and Drug Free Schools (local; state grant portion of the 20% set-
aside; or national)

Substance Abuse Prevention and Treatment (SAPT) Block Grant
State Incentive Grant (SIG) or SPF/SIG Sub-recipient
Traffic Safety Funding (local or state)

 

 

 

 

 

 

If you are a CURRENT Drug-Free Communities Support Program grantee, please
answer the following questions:

238

 

38. What is your DFCSP Grant ID number?

 

39. Please indicate which Cycle you are currently in:
Cycle I (Years 1-5) Cycle II (Years 6 and over)

40. What year did you receive funding for your CURRENT DFCSP grant?

 

 

41. What sources of 42. Please rank the t_op_§
funding/resources largest sources of
support your coalition funding/resources
and its strategies? (THAT YOU
CHECKED IN ITEM

(check all that apply) 41) that support your
coalition and its
strategies.

(1 = highest level of funding,
2 = next highest level of
funding, etc.)

 

Asset forfeiture

 

City government

 

Corporate or Business

 

County government

 

Federal government — Direct from
_Agency

Federal Govemment — Via Block

Grant

Federal Government — Other

 

 

 

Foundations

 

F undraising

 

In-kind contributions

 

Non-profits (e. g., United Way)

 

Private donations

 

State government

 

Sales of products or Dues

Other

(specify)
NONE

 

 

 

 

 

SELECTED OUTCOME INDICATORS

43. Does your coalition collect data on the following youth substance abuse outcome
indicators? (check all that apply)

239

Age of first use
Perception of disapproval by parents
Perception of risk or harm
30-Day Use
Perceived availability

44. What instrument(s) do you use to collect your data? (check all that apply)

American Drug and Alcohol Survey (ADAS)
Communities That Care Survey (CTC) or the CSAP Substance Abuse Risk
and Protective Factor Student Survey (SARPF)
Search Institute’s Developmental Assets Survey
Pride Survey (PRIDE)
Youth Risk Behavior Survey (Y RBS)
Items/Scales from the CSAP Core Measures Initiative
Survey(s) sponsored/developed by your state
(Please specify the name of your state
survey)
Survey(s) that your coalition or community created/developed for your own

purposes
Other (specify)

 

 

Other (specify)

 

45. When did you start collecting survey data in your community?

Year Don’t know/Does not apply

 

 

46. Who is responsible for coordinating/analyzing your survey data? (check all that
apply)

__City

_Coalition
__College/University
__Consultant

_County

_Member organization staff person
__Research organization

__State
_Other (specify)
_Other (specify)
_Don’t know/Does not apply

 

 

47. Does your coalition collect/utilize any of the following social indicator data?
(check all that apply)

240

_Alcohol consumption/sales data
_Alcohol/drug-related traffic fatalities data
__Alcohol outlet density data
_Arrest/incarceration data

Census data
Crime data

_Drug Abuse Warning Network (DAWN) data
_Emergency room data

_Health data

__National Registry of Substance Abuse Treatment Services (formerly the Uniform

Facility Data Set)

__National Registry on Drug Use & Health data (formerly the National Household
Registry on Drug Abuse)

_Other (specify)

 

_Other (specify)

 

__ NONE

 

48. Does your
coalition
collect data
on any of the
following
substances?

(check all that apply)

49.

Based on available, verifiable data
(e. g., Health Department data,
substance abuse incidence/prevalence
data, Emergency Room data, etc.),
which of the following are currently
causing the most problems in your
community?

I = the biggest problem

2 = the next biggest problem

3 = the third biggest problem

4 = fourth biggest problem

5 = fifth biggest problem
RANK 5 ONLY

 

Alcohol

 

Amphetamines

 

Barbituates

 

Cocaine (including Crack)

 

GHB

 

Hallucinogens (other than
LSD)

 

Heroin

 

Hydrocodone

 

Inhalants

 

LSD

 

Marijuana

 

MDMA (Ecstasy)

 

 

 

241

 

Methamphetamine
(including Ice)

 

Nitrites

 

Oxycodone (Oxycontin)

 

PCP

 

Prescription Drugs (other
than Hydrocodone and
Oxycodone)

 

Rohypnol (date rape drug)

 

Steroids

 

Tobacco

 

Tranquilizers

 

Other:

 

 

Other:

 

 

 

242

 

APPENDIX D
2008 CADCA Annual Survey of Coalitions
2008 ANNUAL SURVEY OF COALITION S
Overview

The aim of CADCA’s Annual Survey of Coalitions (formerly the National Coalition
Registry) is to identify coalitions around the country and learn more about what they are
doing in their communities to address substance abuse problems. By adding your voice
to the Annual Survey of Coalitions, you will provide critically needed information that
will advance the coalition field and help inform CADCA of training and technical
assistance needs that are vital to your coalition. The Annual Survey will serve as a single
source where coalitions, policymakers, researchers and practitioners can access up-to-
date information on coalitions. Thank you for taking time to join the movement and
making sure your coalition is counted in CADCA’s Annual Survey of Coalitions!

Different types of audiences will have different types of access to the information in the
Annual Survey. Audiences with access include coalitions who complete the Annual
Survey, CADCA staff, researchers and federal partners. Coalitions who complete the
Annual Survey will have a unique user ID and password so they can access information
collected in the Survey. They will be able to view summary tables pertaining to coalition
demographics, descriptions, strategies and interventions. Survey coalitions that are also
current CADCA members will be able to view all this information plus have access to
customizable reports that will allow them to network with other Annual Survey coalitions
in the country. Finally, as is appropriate, states, federal partners and other government
agencies, researchers and CADCA staff will have access to survey data and results.

General Instructions

CADCA’s Annual Survey of Coalitions will take approximately 30 minutes to complete
and asks questions regarding your coalition’s characteristics, activities and focus. We are
requesting that the person in charge of the day-to-day operations of your coalition
(director, CEO, executive director, etc.) fill out the Annual Survey; however, we
understand that it may make sense for another person to fill out portions of the Annual
Survey.

Please answer Annual Survey fi'om the perspective of the coalition you represent:

0 If your coalition is the umbrella coalition or lead coalition (i. e., coalition of
coalitions), please answer from that perspective — DO NOT ANSWER ON
BEHALF OF THE COALITIONS YOU LEAD.

0 If your coalition is an affiliate within a network of coalitions or a freestanding
coalition, please answer from that perspective.

243

Survey Prizes
Complete* the Annual Survey by December 18, 2008 and your coalition is eligible to

wrn:
0 Free 2009 CADCA National Leadership Forum registration (2 prizes)

Complete* the Annual Survey by January 31, 2009 and your coalition is eligible to win
one of the following prizes:

0 Free 2009 Mid-Year Training Institute registration (2 prizes)

0 Digital camera (2 prizes)

0 Ipod Shuffle (1 prize)

Please read the following and put your initials in the box:

 

I have read the above information, understand the purpose and use of the
information, agree to the above standards about the sharing of my coalition’s
information with the public, and give my consent to participate in the CADCA
Annual Survey of Coalitions.

 

 

 

Notice - The Annual Coalition Survey is administered and maintained by CADCA. The
collection of information for the Annual Coalition Survey is estimated to average 60
minutes per respondent, including the time for reviewing instructions, searching existing
data sources, completing and reviewing the information. Send comments regarding the
Annual Coalition Survey to CADCA, Annual Coalition Survey, 625 Slaters Lane, Suite
300, Alexandria, VA 22314 or contact us at 1-800-54-CADCA or registry@cadca.org.

 

Please check this box if your coalition is no longer active. You do not have to
complete the Annual Coalition Survey but we ask that you contact CADCA
at registrygrcadcanrg or 800-542-2322, ext. 243 to let us know. If your coalition
should resume in 2009, please contact us so you can be counted in the 2009
Survey.

 

 

 

 

*Please note — all items with an asterisk* are required in order for your coalition to
be included in the 2008 Annual Survey of Coalitions.

GENERAL COALITION CONTACT INFORMATION

Coalition Name:

 

 

Mailing Address:

 

City:

 

State:

 

Zip+4 (i.e., 223144212):

 

244

 

 

 

 

 

County: Country:

Phone: ( ) Ext. __

Fax: ( ) Don’t have___
E-mail: Don’t have—
Web site: Don’t have___

 

Is your street address the same as your mailing address?

Yes No

IF YES, PLEASE SKIP TO THE NEXT PAGE
IF NO, PLEASE PROVIDE YOUR STREET ADDRESS BELOW

Street Address (if different from above):

 

City:

 

State:

 

Zip+4: (i.e., 22314-1212):

 

PRIMARY RESPONDENT IN FORMATION

IF YOU ARE NOT THE DIRECTOR OF YOUR COALITION, PLEASE DIRECT
THIS S UR VE Y TO THA T PERSON.

We are requesting that the person in charge of the day-to-day operations of your coalition
(director, CEO, executive director, etc.) fill out the Survey (however, we understand that
it may make sense for another person to fill out portions of the Survey.) Please provide
contact information for the individual who is the PRIMARY respondent.

Name (i.e., Ms. Joan Smith):

 

 

 

Phone: ( ) Ext.
E-mail: Don’t have
What is your title?

 

How long have you held this position?
Start Month Start Year

 

 

245

Are you responsible for the day-to-day operation and leadership of the coalition?
(i. e., you are the coalition leader, executive director, etc.)

Yes No

If you are not the director, what is the name of the director of your coalition (if
applicable)?

 

COALITION DIRECTORY QUESTIONS

3. *Which of the following best describes the focus/activities of your coalition?
(check one only)

_Mission focused solely on Alcohol, Tobacco and Other Drug issues (AT OD
specific)

__ATOD issues as part of a broader mission/array of issues

_No focus on ATOD issues

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

2. *Are any of the following 3. *PLEASE ANSWER IF YOU
organizations/groups represented on your CHECKED YES IN ITEM 2. To what
coalition? degree does the organization/group

contribute to your coalition’s strategic
efforts?
YES NO I = Not at all
2 = Not very much
3 = Somewhat
4 = A gr_eat deal

Alcohol beverage control 1 2 3 4

Alcohol industry 1 2 3 4

Business groups 1 2 3 4

Child protective services 1 2 3 4

Citizen action groups 1 2 3 4

Civic/ fraternal organizations I 2 3 4

Community organizations (e. g., l 2 3 4

grass roots groups, ethnic

organizations, special populations)

Courts 1 2 3 4

Elected officials l 2 3 4

Employment services I 2 3 4

Faith communm' l 2 3 4

Government - county 1 2 3 4

Government - federal (e. g., DEA, l 2 3 4

NHT SA)

 

 

 

 

246

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

2. *Are any of the following 3. *PLEASE ANSWER IF YOU
organizations/groups represented on your CHECKED YES IN ITEM 2. To what
coalition? degree does the organization/group

contribute to your coalition’s strategic
efforts?
YES NO 1 = Not at all
2 = Not very much
3 = Somewhat
4 = A great deal

Government - local 1 2 3 4

Government — state 1 2 3 4

Government - tribal 1 2 3 4

Health 1 2 3 4

Hospital(s) l 2 3 4

Housing authority 1 2 3 4

Human Services 1 2 3 4

Institutions of higher Ieaming 1 2 3 4

Labor 1 2 3 4

Law enforcement 1 2 3 4

Local government executives 1 2 3 4

Mass media 1 2 3 4

Military Services (e.g., Air force, 1 2 3 4

Army, Marines, Navy)

National Guard 1 2 3 4

Parents 1 2 3 4

Prevention providers/centers 1 2 3 4

Private Citizens (other than parents 1 2 3 4

and youth)

Probation 1 2 3 4

Public assistance 1 2 3 4

Public Health 1 2 3 4

Recovery community 1 2 3 4

Recreation department 1 2 3 4

Elementary School 1 2 3 4
30110018 Junior High/ Middle 1 2 3 4
(K42) School

High School 1 2 3 4

State ATOD agency (SSA) 1 2 3 4

Transportation 1 2 3 4

Treatment providers/centers l 2 3 4

Volunteer organizations 1 2 3 4

Youth 1 2 3 4

Other: 1 2 3 4

 

 

 

 

 

247

 

 

 

 

 

 

 

 

 

2. *Are any of the following 3. *PLEASE ANSWER IF YOU
organizations/groups represented on your CHECKED YES IN ITEM 2. To what
coalition? degree does the organization/group

contribute to your coalition’s strategic
efforts?
YES NO I = Not at all
2 = Not very much
3 = Somewhat
4 = A great deal
Other: 1 2 3 4
4. *In the last 12 months, have new members joined your coalition?
Yes No
5. *In the last 12 months, were coalition members replaced by other
representatives (e.g., chief of police steps down from coalition and a new chief
takes his/her place)?
Not at all Not very much Somewhat A great deal
Not applicable
6. *In the last 12 months, to what degree have you re-engaged existing members to
be more actively involved in the coalition?
Not at all Not very much Somewhat A great deal
Not applicable
7. *In the last 12 months, to what degree have you increased the diversity of the
coalition membership (e.g., cultural diversity, sector diversity, etc.)?
Not at all Not very much Somewhat A great deal
Not applicable
8. *To what extent does the coalition demographic composition (e.g., race,
ethnicity, religion, gender, age, sexual orientation, etc.) reflect the demographic
composition of your target community?
Not at all Not very much Somewhat A great
deal
9. *To what extent would you STRONGLY DISAGREE AGREE STRONGLY
disagree or agree with the DISAGREE AGREE
following?

 

Coalition members share a
common vision for the
community.

 

The coalition has an effective
conflict resolution process.

 

 

 

 

 

248

 

 

The coalition has a feeling of
cohesiveness and team spirit.
Coalition members trust each
other sufficiently to honestly
share information, perceptions
and feedback.

The coalition uses collaborative
decision-making processes.

 

 

 

 

 

 

 

10. *Some coalitions focus on programs or services, others focus on changing aspects
of their community environment. Which of the following do you think best
describes the current status of your coalition (choose the answer that best
describes your coalition today):

_ Exclusively providing or coordinating the provision of prevention programs or
services

_ Primarily providing or coordinating the provision of prevention programs, with
some involvement in environmental or policy change.

_ About evenly split between prevention programs/services and environmental
change

__ Primarily focusing on environmental or policy change, with some involvement
in prevention programs and services

__ Exclusively focusing on environmental or policy change.

11. *When was your coalition first started?
Month Year Don’t Know

12. *How does your coalition identify itself (i.e., which definitions best reflect your
coalition regarding scope) (check all that apply)

___Faith-based
__Prevention-focused
__Primari1y serving a special population(s) (e. g., Latino youth, refugee community,
people with disabilities, etc.).
Please specify the special population(s) that you work
with:
__School-based
__Serving an economically disadvantaged community
__Recovery—focused
__Treatment focused
_____Other (specify)
_Other (specify)

 

 

 

13. *Which category BEST describes the geographic area that you target (i. e., where
you focus your effort; population you want to impact)? (check one only)

City

249

__Congressional District
_County

_Indian Reservation
_Neighborhood/Community
_Multiple Counties
__Multiple Neighborhoods
__School District
__School Site
__Multiple School Sites
_Regional
_Statewide

_Other (please specify)

 

 

14. *What best describes your geographic target area? (check one only)

Frontier Rural Suburban Urban
15. Do you target the entire geographic area or a particular sub-population in that
area (i. e., a smaller subset of the geographic area, e. g., at-risk youth,
Hispanic/Latino adults, teenage girls, etc. )?

__Entire Area

__Sub-Population Please identify, as specifically as possible, the
subpopulation(s) targeted by your coalition (e.g., American
Indian, Youth, Hispanic/Latino, etc.):

 

16. *What was your COALITION ’s (not fiscal agent) annual budget for the last
fiscal year?

_Do not have a budget

_$9,999 or less __$1,000,000 to $1,999,999
__$10,000 to $49,999 _$2,000,000 to $2,999,999
__$S0,000 to $99,999 _$3,000,000 to $3,999,999
_$100,000 to $199,999 __$4,000,000 to $4,999,999
_$200,000 to $299,999 _$5,000,000 to $9,999,999
___$300,000 to $399,999 _$10,000,000 to $ 14,999,999
___$400,000 to $499,999 _$15,000,000 to $19,999,999
_$500,000 to $999,999 _$20,000,000 and above

17. *How many full-time equivalent paid STAFF work on alcohol, tobacco or other
drug activities, including in-kind staff (i. e, paid by someone else to work at your
coalition)? (check one only)

None Fewer than 3 3 — 5 6 — 7 8 — 10
11 - 20 More than 20

250

18. *How many VOLUNTEERS on average per month actively participate in your
coalition’s activities/strategies? (Volunteer: someone who does not get paid a salary
for their involvement in your coalition) (check one only)

None 1-5 6-10 1 1-25 26-50
More than 50

19. How would you best describe your coalition? (check one only)

Novice Intermediate Advanced

INSTITUTE TRAININGS, TECHNICAL ASSISTANCE & PUBLICATIONS

20. *Has your coalition had any contact with CADCA or CADCA’s National
Coalition Institute in the last year?

Yes No Don’t know
IF YES or DON’T KNOW, PLEASE CONTINUE ON TO ITEM 21.
IF NO, PLEASE SKIP TO ITEM 27.

21. *What type of contact did your coalition have? (check all that apply)

_CADCA Leadership Forum

__Brief Technical Assistance (via phone or E-mail)
_CADCA Mid-Year Training Institute
_Training

_Membership

_Web site

_Publications

__Personal Coaching

_Distance Ieaming (web-based Ieaming)
_Other (specify)
_Other (specify)

 

 

For items 22-33, please reflect back on the training, technical assistance and
publications yopr coalition pas received from CADCA/CADCA’S National Coalition

Institute in the last 12 months.
22. *Is your coalition using the information you have received from Training,
Technical Assistance or CADCA/Institute publications to inform your decision-

making practices?

Not at all Not very much Somewhat A great deal

251

 

Not applicable

23. *How would you describe your coalition’s effectiveness as a result of training,
technical assistance or CADCA/Institute publications?

Less effective No change More effective
_Not applicable

24. *Has training and technical assistance information been shared with other
people in your coalition?

Not at all Not very much Somewhat A great deal
Not applicable

25. *Are the individuals who were trained/received technical assistance still involved
on your coalition?

_All _Most _Some _None _Not applicable

26. *Did your coalition share training and technical assistance information with
others in your community (outside of the coalition membership)?

Not at all Not very much Somewhat A great deal
Not applicable

Publications

27. *Has your coalition received any communication from CADCA/National
Coalition Institute (e.g., SPF Primers, training announcement, etc.) in the last
year?

Yes No _Don’t know
28. *Has anyone in your coalition read any of the publications (e.g., Assessment

Primer, Capacity Primer, Cultural Competence Primer, Evaluation Primer,
Coalition Handbook, etc.)?

Yes No Not applicable

29. *Has your coalition shared any of the publications (e.g., Assessment Primer,
Capacity Primer, Cultural Competence Primer, Evaluation Primer, Coalition
Handbook, etc.) with other community/coalition members?

Not at all Not very much Somewhat A great deal
Not applicable

252

 

30. *How useful have these publications (e.g., Assessment Primer, Capacity Primer,
Cultural Competence Primer, Evaluation Primer, Coalition Handbook, etc.)
been to your coalition’s work?

Not at all Not very much Somewhat A great deal
Not applicable

For items 31-31, please think back on any of the services and supports (e.g.,
training, technical assistance, publications) your coalition has EVER received from
CADCA/National Coalition Institute.

31. *To what extent is your coalition using the services and supports to inform
decision-making practices in your coalition?

Not at all Not very much Somewhat A great deal
Not applicable

32. *To what extent have the services and supports improved your coalition’s
capacity to effectively address your community’s substance abuse problems?
(check one only)

Not at all Not very much Somewhat A great deal
Not applicable

33. *In general, how satisfied are you with the services and supports your coalition
has received from CADCA/CADCA Institute? (check one only)

_Very satisfied
_Somewhat satisfied
__Somewhat dissatisfied
_Very dissatisfied
_None received

 

 

 

COALITION PROCESSES
34. *Please indicate how your Involvement in Activity in the last 12 months
coalition has been involved (check all that apply)

(either directly or in partnership)

in each of the following activities
in the last 12 months.

 

TYPE OF ACTIVITY Not Initiated Revisited/Modified Completed
done

 

Assess community needs & resources

 

Collected local data on the readiness of
your community/system to select and
implement strategies found to be
effective in addressing substance abuse.

 

 

 

 

 

253

 

34. *Please indicate how your
coalition has been involved
(either directly or in partnership)
in each of the following activities

in the last 12 months.

 

Involvement in Activity in the last 12 months
(check all that apply)

 

TYPE OF ACTIVITY

Not
done

Initiated

Revisited/Modified

Completed

 

Collected local data to determine or
monitor the extent of substance abuse
problems in the community (Needs
Assessment).

 

Collected local data on risk and
protective factor predictors for
substance abuse and other related
problems.

 

Collected information on resources that
are available to address identified
community problems.

 

Collected information on your
community’s history (e.g., major
events/ forces that have affected the
community and/or influenced targeted
outcomes)

 

Defined the geographic boundaries of
the coalition’s target community.

 

Identified the demographics of the
people within the coalition’s target
conununhy.

 

Developed a clear statement or set of
statements that define the true
concerns/problem(s) in the community.

 

Analyze problems and oals

 

Facilitated a group problem analysis to
explore the complex issue of substance
abuse in the community

 

Incorporated the experience and
expertise of coalition members to
thoroughly name and frame problems

and goals

 

Identified local conditions (personal
and environmental) that maintain risk
or build protection in your community.

 

Used data to prioritize substance abuse
needs.

 

Developed a framework or model of change

 

Developed a logic model for what the
coalition will accomplish.

 

Improved the logic model based on new
information about the community and
the coalition’s efforts.

 

Ensured consensus for the logic model

 

Ensured the logic model reflects the
culture and values of the community.

 

 

 

 

 

254

 

 

34. *Please indicate how your
coalition has been involved
(either directly or in partnership)

in each of the following activities
in the last 12 months.

Involvement in Activity in the last 12 months
(check all that apply)

 

TYPE OF ACTIVITY

Not
done

Initiated Revisited/Modified Completed

 

Develop strate ic & action lans

 

Developed population-level
strategies/activities for changing
community conditions and behaviors.

 

Linked coalition assets and resources to
identified community needs.

 

Developed an action plan that identifies
who, will do what, by when to realize
targeted community changes.

 

 

 

 

 

Identify, adapt and implement interventions

 

Matched prioritized substance abuse
needs with evidence-based policies,
program and practices.

 

Developed local responses that employ
multiple strategies in response to
identified needs

 

Prioritized needed community and
systems changes

 

Advocated for the adoption of needed
community and systems change.

 

Developed a plan that ensures a truly
comprehensive response to substance
abuse.

 

 

 

 

 

Evaluate the coalition

 

Developed a local group/committee to
oversee local data collection and/or
analysis

 

Based on the coalition’s logic model,
identified workable measures of success

 

Developed or selected instruments and
methods to assess outcomes.

 

Identified stakeholder concerns that
must be addressed as part of the
evaluation

 

Collected data to monitor the
implementation of programs/strategies.

 

Collected data to assess
immediate/intermediate outcomes of
programs/strategies.

 

Collected data on community changes
(new or expanded polices, programs or
practices brought about by the
coahfion)

 

Collected data to assess long-term
outcomes of programs/strategies.

 

Distributed the work of coalition

 

 

 

 

 

255

 

34. *Please indicate how your
coalition has been involved
(either directly or in partnership)
in each of the following activities
in the last 12 months.

Involvement in Activity in the last 12 months

(check all that apply)

 

TYPE OF ACTIVITY

Not
done

Initiated

Revisited/Modified

Completed

 

evaluation appropriately among
members and staff

 

Made changes to programs/strategies
based on evaluation data.

 

Disseminated coalition evaluation
results and monitoring data to key
community stakeholders.

 

Re-designed evaluation methods and/or
measures to collect better/additional
information.

 

Sustain projects & initiatives

 

Used data as a basis for new grants or
fundingprgosals.

 

Identified what must be sustained.

 

Identified what resources (e. g.,
supplies, space, personnel,
transportation, technical assistance and
funding) are required to sustain
identified needs.

 

Created case statements that clearly
identify what must be sustained, why
and how.

 

Determined how the coalition plans to
provide or develop needed resources to
fund the identified needs.

 

Identified potential partners that will be
needed or recruited to help support each
identified need.

 

Action planned what will be done to
contact potential partners, when each
task will be accomplished and what
resources will be required to contact
and present to potential partners

 

 

 

 

 

256

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

COALITION 35. *How often do you use the 36. *Did you 37. *In the last 12
PLANNING product to inform the decisions of develop months, did you
PRODUCTS your coalition? OR revise develop OR revise
1 = Not at all this this product as a
2 = Not very often product in result of the training
3 = Somewhat the last 12 or technical
4 = Frequently months? assistance you
5 = Always received from the
N/A = Don’t have this product Please circle CADCA Institute?
Please circle your response below your response Please circle your
below response below
Community 1 2 3 4 5 DON’T
Assessment N/ A YES NO YES NO KNOW
Logic Model 1 2 3 4 5 DON’T
N / A YES NO YES NO KNOW
Strategic / l 2 3 4 5 DON’T
Action Plan N/A YES NO YES NO KNOW
Sustainability 1 2 3 4 5 DON’T
Plan N/A YES NO YES NO KNOW
Evaluation 1 2 3 4 5 DON’T
Plan N/A YES NO YES NO KNOW
38. * Please rate the degree to which 1 = Not at all
each product is a comprehensive, 2 = Not very much
effective planning tool for the 3 = Somewhat
coalition? 4 = A great deal

N/A = Don’t have this product
Please circle your response below

 

 

 

 

 

Community Assessment 1 2 3 4 N/A
Mic Model 1 2 3 4 N/A
Strategic / Action Plan 1 2 3 4 N/A
Sustainability Plan 1 2 3 4 N/A
Evaluation Plan 1 2 3 4 N/A

 

 

COALITION STRATEGIES

 

39. *To what extent did your coalition engage in the following strategies in I = Not at all
the LAST 12 MONTHS? 2 = Not very much
3 = Somewhat
4 = A great deal
Please circle your
response below

 

Providing information - Educational presentations, workshops or seminars,

 

and data or media presentations (e. g., public service announcements, brochures, l 2 3 4
billboard campaigns, community meetings, town halls, forums, Web-based

communication).

Enhancing skills - Workshops, seminars or activities designed to increase the

skills of community members, such as youth, parents and citizens (e. g., l 2 3 4

training, parenting classes, evidence-based prevention programs for youth).

Providing support - Creating opportunities that reduce risk or enhance

protection (e.g., providing alternative activities, mentoring, support groups, 1 2 3 4
jouth clubs, parentinggroups, Alcoholics or Narcotics Anonymous).

 

 

 

 

257

 

39. *To what extent did your coalition engage in the following strategies in l = Not at all
the LAST 12 MONTHS? 2 = Not very much

3 = Somewhat
4 = A great deal

Please circle your

response below

Enhancing access/reduce barriers - Improving community and service
delivery processes to increase the ease, ability and opportunity for community I 2 3 4
members to access and use the services (e. g., access to treatment, childcare,
transportation, housing, education, special needs, cultural and language
sensitivity).

 

Changing consequences — Using incentives and disincentives to alter
consequences of a specific behavior (e. g., increasing public recognition for l 2 3 4
deserved behavior, individual and business rewards, taxes, citations, fines,
revocations/loss of privileges).

 

Changing the physical design of the environment - Altering the physical
structure of the community so that individuals are less likely to engage in 1 2 3 4
substance use (e. g., parks, landscapes, sigpage, lighting, outlet density).

 

 

Modifying/changing policies — Working to create formal changes in written
procedures, by-laws, proclamations, rules or laws with written documentation 1 2 3 4
and/or voting procedures (e. g., public policy actions, systems change within
government, communities and organizations).

 

 

COMMUNITY CHANGES

Policy/Practice Change

40. *In the last 12 months, did your coalition help bring about a NEW or
MODIFIED policy or practice change (e.g., coalition helps merchants display
“We Card” signs or coalition facilitates the development and passing of a city
ordinance that requires Responsible Beverage Server Training for all alcohol
retailers in the city)?

Yes No

IF YES, PLEASE CONTINUE ON TO ITEM 41.
IF NO, PLEASE SKIP TO ITEM 44.

41. *In the last 12 months, approximately how many NEW or MODIFIED policy or
practice changes did your coalition help bring about?

number

42. *In which sectors were you able to bring about the policy/practice changes?
(Check all that apply)

_Nonprofit

Business
Govemment/law

258

_Education/school
_Community
_Health

_Social services
_Media

__Family and caregiver
__Religious

_Youth

43. *Who were the targets of these policy/practice changes? (check all that apply)

__Youth

_Parents and caregivers

_Families

_Community leaders .
_General community

Programmatic Change

44. *In the last 12 months, did your coalition help bring about a NEW or
MODIFIED program (e.g., coalition helps partner organization select and
implement a new parenting program or coalition works with partner to modify
existing program by providing a Spanish-language version)?

Yes No
IF YES, PLEASE CONTINUE ON TO ITEM 45.
IF NO, PLEASE SKIP TO ITEM 48.

45. *In the last 12 months, approximately how many NEW or MODIFIED
programs did your coalition help bring about?

number

46. *In which sectors were you able to bring about the program changes? (Check all
that apply)

_Nonprofit
_Business
__Govemment/ law
__Education/school
_Community
_Health

_Social services
__Media

__Family and caregiver

259

__Religious
_Youth

47. *Who were the targets of these programmatic changes? (check all that apply)

_Youth

__Parents and caregivers
__Families
_Community leaders
__General community

Please reflect on all the community changes (i.e., new or modified programs,
policies, and practices) your coalition has helped facilitate in the past 12 months.

48. *To what extent are these community changes addressing the reduction of
targeted substance abuse problems?

Not at all Not very much Somewhat A great deal
Not applicable

49. *To what extent are these community changes addressing community
risk/protective factors associated with your targeted substance abuse problems?

Not at all Not very much Somewhat A great deal
Not applicable

50. *To what extent are these community changes addressing local conditions that
contribute to your targeted substance abuse problems?

Not at all Not very much Somewhat A great deal
Not applicable

Other Community Changes

51. *To what extent are your community’s norms shifting to support substance
abuse reduction efforts?

Not at all Not very much Somewhat A great deal
52. *To what extent has your coalition played a role to shift these norms?
Not at all Not very much Somewhat A great deal

53. *To what degree is your coalition meeting the unique demographic and cultural
needs of your community?

260

Not at all Not very much Somewhat A great deal

54. *Is your coalition meeting with other organizations to share ideas and get buy-in
to the work your coalition is doing?

Not at all Not very much Somewhat A great deal

 

55. *Overall, how effective do you think your coalition is at the I = Ineffective
following? 2 = Somewhat effective

3 = Effective

4 = Extremely effective

Please circle your response

 

 

 

 

 

 

 

 

 

below
Working strateggzallL l 2 3 4
Making decisions. 1 2 3 4
Working comprehensively to address your local substance abuse 1 2 3 4
problems.
Resolving conflict. 1 2 3 4
Getting members engaged in the work. 1 2 3 4
Reducing population-level substance abuse problems. 1 2 3 4
56. *To what extent do you think They would They They would They would
that your community STRONGLY would DISAGREE STRONGLY
members (outside the AGREE AGREE DISAGREE

coalition membership) agree
with the following:

 

The coalition is a community
change agent.

The coalition is a community
problem solver.

The coalition is a service provider.
The coalition is focused on
environmental strategies.

The coalition is focused on
individual-level change initiatives.
The coalition is focused on
population-level reductions in
community substance abuse
problems.

 

 

 

 

 

 

 

 

 

 

261

 

SUSTAINAflLITY

 

57. *What 58. Please rank the _tpp_5 59. *Did you
sources of largest sources of get a new
funding/resou funding/resources source of
rces support (THAT YOU funding in
your coalition CHECKED IN ITEM one of
and its 41) that support your these
strategies? coalition and its categories

strategies. in the last

(check all that year?

apply) (1 = highest level of funding,

2 = next highest level of (check all that
funding etc.) apply)

 

Asset forfeiture

 

City government

 

Corporate or Business

 

County government

 

Federal government

 

Foundations

 

Fundraising

 

In-kind contributions

 

Non-profits (e.g., United
Way)

 

Private donations

 

State government

 

Sales of products or Dues

 

Other
(Specify)

 

NONE

 

 

 

 

60. Does your coalition have a fiscal agent (i. e., an organization/entity that administers

 

 

your funds)?
Yes No
61. Has your coalition been granted nonprofit status by the IRS (e. g., has 501 (c)3
status)?
Yes No
62. Has your coalition received funding from any of the following Currently In the past
programs? (2008-2009) (Before .
(check all that apply) 2008)
Drug Free Communities Support Program

 

 

 

262

 

Drug Free Communities Support Program Mentoring Grant

 

Community Capacity and Development Office (formerly the Executive
Office of Weed and Seed)

 

Office of Juvenile Justice and Delinquency Prevention (OJJDP) Underage
Drinking Initiative

 

Safe and Drug Free Schools (local; state grant portion of the 20% set-aside;
or national)

 

Substance Abuse Prevention and Treatment (SAPT) Block Grant

 

SPF/SIG Sub-recipient

 

STOP Act Grant

 

 

 

Traffic Safety Funding (local or state)

 

63. Does your coalition obtain training and technical assistance, materials, and/or
logistical support from any of the following? (check all that apply)

__Centers for Application of Prevention Technologies (CAPTs)
__Centers for Disease Control and Prevention (CDC)

_Center for Substance Abuse Prevention (CSAP)

__Center for Substance Abuse Treatment (CSAT)
_Departments/Systems in your state

_Enforcing the Underage Drinking Laws (EUDL) Program (OJJDP/PIRE)
_National Clearinghouse for Alcohol & Drug Information (NCADI)
__National Guard

_National Highway Traffic Safety Administration (NHTSA)
_National Institute on Alcohol Abuse and Alcoholism (NIAAA)
__National Institute on Drug Abuse (NIDA)

_NONE

COALITION INlfiCATORS AND ACTIVITIES

64. *Does your coalition collect data on the following youth substance abuse
outcome indicators? (check all that apply)

Age of first use

Perception of disapproval by parents
Perception of risk or harm

30-Day Use

Perceived availability

65. *Does your coalition collect/utilize any of the following social indicator data?
(check all that apply)

Alcohol consumption/sales data

:Alcohol/drug-related traffic fatalities data
_Alcohol outlet density data

263

_Arrest/incarceration data

_Census data

__Crime data

_Drug Abuse Warning Network (DAWN) data

_Emergency room data

_Health data .

_National Registry of Substance Abuse Treatment Services (formerly the Uniform
Facility Data Set)

_National Registry on Drug Use & Health data (formerly the National Household

Registry on Drug Abuse)
_Other (specify)

 

_Other (specify)

 

_ NONE

66. What instrument(s) do you use to collect your student survey data? (check all
that apply)

American Drug and Alcohol Survey (ADAS)
Communities That Care Survey (CTC) or the CSAP Substance Abuse Risk
and Protective Factor Student Survey (SARPF)
Search Institute’s Developmental Assets Survey
Pride Survey (PRIDE)
Youth Risk Behavior Survey (Y RBS)
Items/Scales from the CSAP Core Measures Initiative
Survey(s) sponsored/developed by your state
(Please specifi» the name of your state
survey)
Survey(s) that your coalition or community created/developed for your own
purposes
Other (specify)

 

 

Other (specify)

 

67. When did you start collecting survey data in your community?

Year Don’t know/Does not apply

 

 

68. Does your coalition have an evaluator (paid or volunteer)?

Yes No

264

 

69. Does your
coalition
collect data on
any of the
following
substances?

(Check all that
apply)

70. Based on available,

verifiable data (e.g., Health
Department data, substance
abuse incidence/prevalence
data, Emergency Room
data, etc. ), which of the
following are currently
causing the most problems
in your community?

1 = the biggest problem

2 = the next biggest
problem

3 = the third biggest
problem

4 = fourth biggest problem
5 = fifth biggest problem

 

71. Please
indicate
which of the
following
substances
your coalition
is currently
address?

(Check all that
apply)

 

Alcohol

 

Amphetamines

 

Barbiturates

 

Binge Drinking

 

Cocaine (including
Crack)

 

Cough Medicine
(DXM)

 

GHB

 

Hallucinogens (other
than LSD)

 

Heroin

 

Hydrocodone

 

Inhalants

 

LSD

 

Marijuana

 

MDMA (Ecstasy)

 

Methamphetamine
(including Ice)

 

Nitrites

 

Oxycodone
(Oxycontin)

 

PCP

 

Prescription Drugs
(other than
Hydrocodone and
Oxycodone)

 

Rohypnol (date rape
drug)

 

 

 

 

265

 

Steroids
Tobacco
Tranquilizers
Other:

Other:
NONE

 

 

72. Please indicate if your coalition was involved in any of the following
interventions in the past 12 months.

YES

NO

 

Media public awareness SPECIFICALLY for ALCOHOL problem reduction, such
as: posters, public service announcements, advertisements, commercials and
billboard campaigns

 

Media public awareness for substance abuse problem reduction, such as: posters,
public service announcements, advertisements, commercials and billboard
campaigns

 

Recovery Month Activities

 

National Medicine Abuse Awareness Month Activities

 

Town hall meetings

 

Media advocacy, media literacy and social marketing activities linked
SPECIFICALLY to ALCOHOL problem reduction such as: counter marketing or
counter-advertising.

 

Media advocacy, media literacy and social marketing activities linked to substance
abuse reduction

 

Increasing access to treatment

 

Recovery support activities such as: sober housing, peer support, relapse prevention
or employment.

 

Re-entry support activities for offenders returning from incarceration

 

Screening, brief intervention and referral to treatment (SBIRT) activities such as
substance abuse assessment in the emergency room, primary care offices, etc.

 

Increasing effectiveness of substance abuse treatment

 

Development of workplace programs or policies to address substance abuse

 

Increasing access to services (e.g., providing or improving transportation and
childcare)

 

Improving cultural and language sensitivity of activities/ services provided in the
community (e.g., translating brochures, offering services/activities in other
languages, etc.)

 

“Shoulder-tap” enforcement program

 

Compliance checks for alcohol or tobacco sales to underage individuals

 

License revocation for tobacco or alcohol sales violations

 

Increased fines for tobacco and alcohol sales violations

 

Increased checks for fake IDs

 

Enforcement of community laws and policies that discourage substance abuse, such
as: neighborhood watch and increased enforcement of underage drinkinglaws)

 

Community policing programs or services, for example: officers who are out in the
neighborhood working with residents on crime prevention

 

Rewards/incentives for deserved behavior (e.g., public recognition of retailers that
pass compliance checks, business/individual rewards, etc.)

 

 

 

266

 

72. Please indicate if your coalition was involved in any of the following
interventions in the past 12 months.

YES

NO

 

Alcohol and cigarette marketing/advertising restrictions in public places

 

Limiting or changing alcohol or tobacco product placement in retail outlets

 

Limitation and restrictions of location and density of alcohol outlets

 

Neighborhood improvement such as: improved lighting in public places or
landscaping

 

Limiting the number of alcohol licenses issued

 

Restrictions on alcohol and/or tobacco use at community events

 

Responsible beverage service training

 

Development of community laws and policies that discourage substance abuse such
as: restricting amounts of problem Over the Counter drugs (OTC) sold or policies to
place goblem OTC s behind counters

 

Implementation ofa social host ordinance

 

Implementing keg registration policies

 

Implementing DUI checkpoints

 

Worked with other entities or coalitions to change state and/or national policies such
as: increasing tax on alcohol/tobacco and reallocation of fines to prevention for
alcohol and tobacco sales violations

 

Systems change within government, communities and organizations (e.g.,
policy/practice changes within community institutions such as schools and
businesses)

 

 

 

MEMBERSHIP
73. *Are you a current member of CADCA?

Yes No _I don’t know

IF YES, PLEASE CONTINUE ON TO ITEM 74.
IF NO or DON’T KNOW, PLEASE SKIP TO ITEM 76.

74. *How satisfied are you with your CADCA membership? (check one)

__Very satisfied
_Satisfied
_Somewhat satisfied
_Somewhat dissatisfied
_Dissatisfied

75. *What are the primary reasons you are a CADCA member? (Please rank your
top 5 in order of importance with 1 = most important and 5 = least important)

Advocacy and public policy information
Discounts and scholarship opportunities
National Leadership Forum and Mid—Year Training Institute

267

 

__The National Community Anti-Drug Coalition Institute
__Professional development for me and my staff.
_Networking opportunities provided.

_CADCA provides national representation for coalitions
_Af’finity programs (e. g. Office Depot discounts; J ZA Insurance)

76. If you answered NO or I DON’T KNOW to QUESTION 73, would you be
interested in receiving information about the benefits of membership?

Yes No

77. What CADCA communications tools do you utilize? (check all that apply)

__CADCA'S Web site, www.cadca.org
__CapWiz for public policy

__Coalitions Online, CADCA's weekly e-newsletter
__Strategizer and Practical Theorist publications
__Satellite broadcasts/webcasts

78. What values do you think CADCA represents most? (check only three)

1:] ambition a trustworthy C] authentic Cl equality

E1 integrity Cl service E1 responsibility CI respect

[:1 dedication C] diversity 13 improvement Cl fun

CI dynamic Cl credibility c1 honesty c1 innovative

El teamwork c1 excellence El accountability Cl empowerment
[:1 high quality [:1 efficiency a strong El collaboration

[:1 reliable [:1 accomplishment E1 distinctive El independence
Cl challenge 1:! influence C] learning D compassion

[:1 simple Cl persistency E] optimism D dependability

E1 flexibility

268

REFERENCES

269

 

REFERENCES

Allen, N. E. (2001). Coordinating councils as vehicles for achieving a coordinated
community response to domestic violence: An examination of the correlates of
council eflectiveness. Allen,-Nicole-Elizabeth: Michigan State U., US.

Allen, NE. (2006). An examination of the effectiveness of domestic violence
coordinating councils. Violence Against Women, 12(1), 46-67.

Allen, N. E. (2005). A multi-level analysis of community coordinating councils.
American Journal of Community Psychologi, 35(1/2), 49-63.

Allen, N. E., Watt, K. A., & Hess, J. Z. (2008). A qualitative study of the activities and
outcomes of domestic violence coordinating councils. American Journal of
Community Psychology, 41 (1-2), 63-73.

Anderson-Butcher, D., Newsome, W.S., & Ferrari, TM. (2003). Participation in Boys

and Girls Clubs and relationships to youth outcomes. Journal of Community
Psychology, 31(1), 39-55.

August, G. J., Lee, S. S., Bloomquist, M. L., Realmuto, G. M., & Hektner, J. M. (2004).
Maintenance effects of an evidence-based prevention innovation for aggressive
children living in culturally diverse urban neighborhoods: The Early Risers

Effectiveness Study [Electronic version]. Journal of Emotional and Behavioral
Disorders, 12(4), 194-205.

Baldwin, T. T., & Ford, J. K. (1988). Transfer of training: A review and directions for
fiiture research. Personnel Psychology, 41, 63-105.

Basen-Engquist, K., O’Hara-Tompkins, N., Lovatto, C.Y., Lewis, M.J., Parcel, G.S., &
Gingiss, P. (1994). The effect of two types of teacher training on implementation

of Smart Choices: A tobacco prevention curriculum. The Journal of School
Health, 64(8), 334-339).

Bickman, L., Andrade, A. R., & Lambert, E. W. (2002). Dose response in child and
adolescent mental health services [Electronic version]. Mental Health Services
Research, 4(2), 57-70.

Bronfenbrenner, U. (1979). The ecology of human development: Experiments by nature
and design. Cambridge, MA: Harvard University Press.

Burke, L. A. & Baldwin, T.T. (1999). Workforce training transfer: A study of the effect

of relapse prevention training and transfer climate. Human Resource
Management, 38(3), 227-242.

270

Butterfoss, F .D. (2004). The coalition technical assistance and training framework:

Helping community coalitions help themselves. Health Promotion and Practice,
5(2), 1 18-126.

Butterfoss, RD. (2007). Coalitions and partnerships for community health. San
Francisco, CA: Jossey-Bass.

Butterfoss, F ., Goodman, R. & Wandersman, A. (1993). Community coalitions for
prevention and health promotion. Health Education Research, 8(3), 315-330.

Butterfoss, F .D., Morrow, A.L., Webster, J .D., & Crews, RC. (2003). The Coalition
Training Institute: Training for the long haul. Journal of Public Health
Management Practice, 9(6), 522-529.

Calabrese, E. J., & Baldwin, L. A. (2001). U-shaped dose-responses in biology,
toxicology, and public health [Electronic version]. Annual Review of Public
‘ Health, 22, 15-33.

Camevale Associates (2008). Federal Drug Budget Trend. Policy Brief Retrieved from
http://carnevaleassociates.com/publications.html.

Center for Substance Abuse Prevention (2009). Strategic Prevention Framework State
Incentive Grant Program Request for Application (RF A No. SP-O9-Ol 1).
Rockville, MD: Substance Abuse Mental Health Services Administration.

Charming Bete. (2001). Communities That Care. Retrieved March 23, 2009, from
http://wch.uhs.wisc.edu/1 3-
Eval/Tools/Resources/ Model%2OPrograms/Communities%20That%20Care. pd f.

 

 

Chaput, S. S., Little, P. M. D., & Weiss, H. (2004). Understanding and measuring
attendance in out-of-school time programs [Electronic version] (No. OSTB7).
Cambridge, MA: Harvard Family Research Project, Harvard Graduate School of
Education.

Charlebois, P., Brendgen, M., Vitaro, F ., Norrnandeau, S., & Boudreau, J .-F . (2004).
examining dosage effects on prevention outcomes: Results from a multi-modal
longitudinal preventive intervention for young disruptive boys. Journal of School
Psychology, 42, 201-220.

Chavis, DM. (2001). The paradoxes and promise of community coalitions. American
Journal of Community Psychology, 29(2), 309-320.

Collie-Akers VL, Fawcett SB, Schultz JA, Carson V, Cyprus J, Pierle J E. (July 2007).

Analyzing a community-based coalition’s efforts to reduce health disparities and
the risk for chronic disease in Kansas City, Missouri. Preventing Chronic Disease

271

[serial online]. 2007 Jul. Available from
http://www.cdc.gov/pcd/issucs/2007/ju1/06_()101 .htrn.

 

Collins, D., Knowlton, J ., & Becker, 8.]. (2007). A meta-analysis of direct and mediating
effects of community coalitions that implemented science-based substance abuse
prevention interventions. Substance Use & Misuse, 42(6), 985-1007.

COMMIT Research Group (1995). Community Intervention Trial for Smoking Cessation
(COMMIT): l. Cohort results from a four-year community intervention. American
Journal of Public Health, 85(2), 183-192.

Dane, A.V. & Schneider, B.H. (1998). Program integrity in primary and early secondary
prevention: Are implementation effects out of control? Clinical Psychology
Review, 18(1), 23-45.

Department of Health and Human Services. (2009). Drug Free Communities Support
Program Request for Application (RFA No. SP-09-002). Washington, DC:
Department of Health and Human Services.

Department of Justice. (2009). CCDO FY 09 Weed and Seed Communities Competitive
Program Guide and Application Kit (Funding Opportunity No. CCDO-2009-
1825). Washington, DC.

Durlak, J. A. & DuPre, E. P. (2008) Implementation matters: A review of research on the
influence of implementation on program outcomes and the factors affecting
implementation. American Journal of Community Psychology, 41(3-4), 327-350.

F awcett, S. B., Francisco, V. T. & Schultz, J. A. (2004). Understanding and improving
the work of community health and development. In J. Burgos & E. Ribes (Eds.),
Theory, basic and applied research, and technical applications in behavior
science. (pp. 209-242). Guadalajara, Mexico: Universidad de Guadalajara.

Fawcett, S., Lewis, R., Paine-Andrews, A., Francisco, V., Richter, K., Williams, E. &
Copple, B. (1997). Evaluating community coalitions for prevention of substance
abuse: The case of Project Freedom. Health Education and Behavior, 24(6), 812-
828.

F awcett, S.B., Paine-Andrews, A., Francisco, V.T., Schultz, J .A., Richter, K.P, Berkley-
Patton, J ., Fisher, J ., Lewis, R.K., Lopez, C.M., Roussos, S., Williams, E.L.,
Harris, K.J., & Evensen, P. (2001). Evaluating community initiatives for health
and development. In I. Rootman, D. McQueen, et al. (Eds), Evaluation in health
promotion. (pp. 241-277). Copenhagen, Denmark: World Health Organization -
Europe.

Fawcett, S.B., Sterling, T.D., Paine-Andrews, A., Harris, K.J., Francisco, V.T., Lewis,
R.K., & Scmid, TL. (1995). Evaluating community eflorts to prevent

272

cardiovascular disease. Atlanta, GA: Centers for Disease Control and Prevention,
National Center for Chronic Disease Prevention and Health Promotion.

Francisco, V.T., Paine, A.l., & Fawcett, SB. (1993). A methodology for monitoring and
evaluating community health coalitions. Health Education Research: Theory and
Practice, 8(3), 403 -416.

F eighery, E., & Rogers, T. (1989). Building and maintaining effective coalitions.
Published as Guide No. 12 in the series How-To Guides on Community Health
Promotion. Stanford Health Promotion Resource Center, Palo Alto, CA.

Feinberg, M.E., Greenberg, M.T., Osgood, D.W., Anderson, A., & Babinksi, L. (2002).
The effects of training community leaders in prevention science: Communities
That Care in Pennsylvania. Evaluation and Program Planning, 25, 245-259.

Feinstein, L. & Hammond, C. (2004). The contribution of adult learning to health and
social capital. Oxford Review of Education, 30(2), 199-221.

F iester, L. M., Simpkins, S. D., & Bouffard, S. M. (2005). Present and accounted for:
Measuring attendance in out-of—school-time programs. New Directions for Youth
Development, 105, 91-107.

Florin, R, Mitchell, R., & Stevenson, J. (1993). Identifying training and technical
assistance needs in coalitions: a developmental approach. Health Education
Research: Theory & Practice, 8, 417-432.

F oster-F ishman, P.G., Berkowitz, S.L., Lounsbury, D.W., Jacobson, S., & Allen, NA.
(2001). Building collaborative capacity in community coalitions: A review and

integrative framework. American Journal of Community Psychology, 29(2), 241 -
261.

Foster-Fishman, P.G., Law, K., Ahn, S. (2008). CADCA National Training Institute
evaluation findings 2006-200 7. Community Anti-Drug Coalitions of America.

Foster-Fishman, P.G., Nowell, B., Archer, G., & Greenway, N. (2005). The National

Community Anti-Drug Coalition Institute: A Formative Evaluation of the
Institute ’s Activities fi‘om September 2003-0ctober 2004. Community Anti-Drug
Coalitions of America.

F oster—F ishman, P.G., White, L., & Droege, E. (2007). The National Community Anti-
Drug Coalition Institute: A formative evaluation of the Institute ’3 activities from
September 2005-October 2006. Community Anti-Drug Coalitions of America.

Green, L. & Kreuter, M. (2003). Fighting Back or fighting themselves? Community

coalitions against substance abuse and their use of best practices. American
Journal of Preventive Medicine, 23 (4), 303-306.

273

 

Guide to Community Preventive Services. (2001). Reducing alcohol-impaired driving:
multicomponent interventions with community mobilization

www.thecommunityguide.org/mvoi/multicomponent.html. Last updated:
02/10/2009.

 

Hallfors, D., Cho, H., Livert, D., & Kadushin, C. (2002). Fighting back against substance
abuse: Are community coalitions winning? American Journal of Preventive
Medicine, 23(4), 237-245.

Hansen, N. B., Lambert, M. J ., & Forma, E. M. (2002). The psychotherapy dose-response
effect and its implications for treatment delivery services [Electronic version].
Clinical Psychology: Science and Practice, 9(3), 329—343.

Hays, C.E., Hays, S.P., DeVille, J .O., & Mulhall, PF. (2000). Capacity for effectiveness:
The relationship between coalition structure and community impact. Evaluation
and Program Planning, 23, 373-379.

Harwood, H.J., Fountain, D., & Livermore, G. (1999). Cost estimates for alcohol and
drug abuse. Addiction, 94(5), 631-647.

Hendry, C (1996). Understanding and creating whole organizational change through
Ieaming theory. Human Relations, 49(5) 621-41 .

Hingson, R.W., Zakocs, R.C., Winter, M.R., Rosenbloom, D., & DeJong, W. (2005).
Effects on alcohol related fatal crashes of a community based initiative to increase

substance abuse treatment and reduce alcohol availability. Injury Prevention, 11,
84-90.

Holcomb, D. L., Smith, M. A., Ware, G. O., Hung, Y.-C., Brackett, R. E., & Doyle, M. P.
(1999). Comparison of six dose-response models for use with food-borne
pathogens [Electronic version]. Risk Analysis, 19(6), 1091-1100.

Howard, K. 1., Kopta, S. M., Krause, M. S., & Orlinsky, D. E. (1986). The dose-effect
relationship in psychotherapy [Electronic version]. American Psychologist, 41(2),
159-164.

Huhman, M. E., Potter, L. 8., Duke, J. C., Judkins, D. R., Heitzler, C. D., & Wong, F. L.
(2007). Evaluation of a national physical activity intervention for children
VERBTM Campaign, 2002—2004 [Electronic version]. American Journal of
Preventive Medicine, 32(1), 38-43.

Institute of Medicine. (2002). Chapter on “Community”. (Pp. 178-211). T he future of the
public ’s health in the 21‘" century. Washington, DC: National Academies Press.

Johnston, L. D., O'Malley, P. M., Bachman, J. G., & Schulenberg, J. E. (December 11,
2008). "Various stimulant drugs show continuing gradual declines among teens in

274

2008, most illicit drugs hold steady. " University of Michigan News Service: Ann
Arbor, MI. Retrieved 03/05/2009 from http://www.monitoringthcfuture.org

 

Julian, DA. (1997). The utilization of the logic model as a system level planning and
evaluation device. Evaluation and Program Planning, 20(3), 251-257.

Kegler, M.C., Twiss, J.M., & Look, V. (2000). Assessing community change at multiple
levels: The genesis of an evaluation fiamework for the California Healthy Cities
Project. Health Education & Behavior, 27(6) 760-779.

Kreuter, M., Lezin, N. & Young, L. (2000). Evaluating community-based collaborative
mechanisms: Implications for practitioners. Health Promotion Practice, 1(1), 49-
63.

KU Work Group for Community Health and Development. (2007). Use Promising
Approaches: Implementing Best Processes for Community Change and
Improvement. Lawrence, KS: University of Kansas. Retrieved November 12,
2008, from the World Wide Web: http://ctb.ku.equ/en/prpmisipgapproaghj.

 

Lasker, R.D. & Weiss, ES. (2003). Broadening participation in community problem
solving: A multidisciplinary model to support collaborative practice and research.
Journal of Urban Health: Bulletin of the New York Academy of Medicine, 80(1),
14-60.

Lasker, R.D., Weiss, E.S., & Miller, R. (2001). Partnership synergy: A practical
framework for studying and strengthening the collaborative advantage. The
Milbank Quarterly, 79(2), 179-205).

Liaw, F .-r., Meisels, S. J., & Brooks-Gunn, J. (1995). The effects of experience of early
intervention on low birth weight, premature children: The Infant Health and
Development Program [Electronic version]. Early Childhood Research Quarterly,
10, 405-431. '

Marsh, H.W., & Kleitman, S. (2002). Extracurricular school activities: The good, the bad,
and the nonlinear. Harvard Educational Review, 72, 464-514.

May, S., & Bigelow, C. (2005). Modeling nonlinear dose-response relationships in
epidemiologic studies: Statistical approaches and practical challenges. Dose-
Response, 3, 474-490.

McCall, R. B., Ryan, C. S., & Plemons, B. W. (2003). Some lessons learned on
evaluating community-based, two-generation service programs: The case of the
Comprehensive Child Development Program [Electronic version]. Journal of
Applied Developmental Psychology, 24(2), 125-141.

McHargue, SK. (2003). Learning for performance in nonprofit organizations. Advances

275

 

in Developing Human Resources, 5(2), 196-204.

McKnight, P.E., McKnight, K.M., Sidani, S., & Figueredo, AJ. (2007). Missing Data: A
Gentle Introduction. New York: The Guilford Press.

Merzel, C. & D’Affitti, J. (2003). Reconsidering community-based health promotion:
Promise, performance, and potential. American Journal of Public Health, 93,
557-574.

Nation, M., Crusto, C., Wandersman, A., Kumpfer, K. L., Seybolt, D., Morrisey-Kane,
E., et a1. (2003). What works in prevention: Principles of effective prevention
programs. American Psychologist, 58(6-7), 449-456.

National Highway Traffic Safety Administration (2008). Statistical Analysis of Alcohol-
Related Driving Trends, 1992-2005. (NHTSA Technical Report, DOT HS 810
942). Washington, DC.

Nowell, BL. (2005). The role of social capital in interorganizational alliances. Nowell,-
Branda: Michigan State U., US.

Office of National Drug Control Policy (ONDCP). (2007). Annual Findings Report 2006:

Drug-Free Communities Support Program National Evaluation. Columbus, OH:
Battelle.

Office of National Drug Control Policy (ONDCP). (2008). Interim DF C Program
Evaluation Findings Report. Columbus, OH: Batelle.

Paine-Andrews, A., Fisher, J ., Berkely-Patton, J ., Fawcett, S.B., Williams, E., Lewis, R.,
Harris, K. (2002). Analyzing the Contribution of Community Change to
Population Health Outcomes in an Adolescent Pregnancy Prevention Initiative.
Health Education & Behavior, 29(2). 183-193.

Paradise, A. (2007). State of the industry report. American Society for Training &
Development.

Quercia, R. G., & Galster, G. C. (2000). Threshold effects and neighborhood change
[Electronic version]. Journal of Planning Education and Research, 20(2), 146-
162.

Quinby, R.K., Hanson, K., Brooke-Weiss, B., Arthur, M.W., & Hawkins, J .D. (2008).
Installing the Communities That Care prevention system: Implementation
progress and fidelity in a randomized controlled trial. Journal of Community
Psychology, 36(3), 313-332.

Resnicow, K., Braithwaite, R., Dilorio, C., Vaughan, R., Cohen, M. I., & Uhl,'G. A.

(2001). Preventing substance use in high risk youth: Evaluation challenges and
solutions [Electronic version]. Journal of Primary Prevention, 21(3), 399-415.

276

Rhodes, J ., Lok, P., Hung, R.Y-y., & Fang, S-c. (2008). An integrative model of
organizational Ieaming and social capital on effective knowledge transfer and

perceived organizational performance. Journal of Workplace Learning, 20(4),
2008.

Rivera, R.J. & Paradise, A. (2006). State of the industry in leading enterprises: AST D ’3
annual review of trends in workplace learning and performance. American
Society for Training & Development.

Rogers, E. M. (2003). Diffusion of innovation (5’11 ed.). New York: Free Press.

Roussos, S.T. & F awcett, SB. (2000). A review of collaborative partnerships as a
strategy for improving community health. Annual Review of Public Health, 21 ,
369-402.

Saks, AM. (1996). The relationship between the amount and helpfulness of entry training
and work outcomes. Human Relations, 49(4), 429-451.

Saks, A.M. & Belcourt, M. (2006). An investigation of training activities and transfer of
training in organizations. Human Resource Management, 45(4), 629-648.

Saxe, L., Reber, E., Hallfors, D., Kadushin, C., Jones, D., Rindskopf, D., & Beveridge, A.
(1997). Think globally, act locally: Assessing the impact of community-based
substance abuse prevention. Evaluation and Program Planning, 20 (3), 357-366.

Schafer, J .L & Graham, J.W. (2002). Missing data: Our view of the state of the art.
Psychological Methods, 7(2), 147-177.

Senge, P. (1990). The fifth discipline: The art & practice of the learning organization.
New York: Doubleday.

Shortell, S.M., Zukoski, A.P., Alexander, J .A., Bazzoli, G.J., Conrad, J .A., Husnain-
Wynia, R., Sofaer, 8., Chan, B.Y., Casey, E., & Margolin, F .S. (2002). Evaluating
partnerships for community health improvement: Tracking the footprints. Journal
of Health Politics, Policy and Law, 27(1), p. 49-92.

Snyder, W.M. & Cummings, TO. (1998). Organization Ieaming disorders: Conceptual
model and intervention hypotheses. Human Relations, 51(7), 873-895.

Sorensen, G., Emmons, K., Hunt, M.K., & Johnston, D. (1998). Implications of the

results of community intervention trials. Annual Review of Public Health, 19,
379-416.

Stevens, G.L. & Lodl, K.A. (1999). Community coalitions: Identifying changes in
coalition members as a result of training. Journal of Extension, 37(2), 1-9.

277

Stevenson, J .F . & Mitchell, RE. (2003). Community-level collaboration for substance
abuse prevention. The Journal of Primary Prevention, 23(3), 371-404.

Substance Abuse and Mental Health Services Administration, Office of Applied Studies
(2008). Results from the 2007 National Survey on Drug Use and Health: National
Findings (NSDUH Series H-34, DHHS Publication No. SMA 08-4343).
Rockville, MD.

Tabachnick, B. G. & Fidell, L. S. (2007). Using multivariate statistics (5th ed.). Boston:
Pearson Education, Inc.

Tannenbaum, S. I. (1997). Enhancing continuous Ieaming: Diagnostic findings from
multiple companies. Human Resource Management, 36(4), 437-452.

Valentine, J ., Gottlieb, B., Keel, S., Griffith, J ., & Ruthazer, R. (1998). Measuring the
effectiveness of the Urban Youth Connection: The case for dose-response
modeling to demonstrate the impact of an adolescent substance abuse prevention
program [Electronic version]. Journal of Primary Prevention, 18(3), 363-3 87.

Watson-Thompson, J ., Fawcett, SB, and Schultz, J.A., (2008). Differential Effects of
Strategic Flaming on Community Change in Two Urban Neighborhood
Coalitions. American Journal of Community Psychology, 42, 25-38.

Weiss, E.S., Anderson, R.M., & Lasker, RD. (2002). Making the most of collaboration:
Exploring the relationship between partnership synergy and partnership
functioning. Health Education & Behavior, 29(6), 683-698.

Welsh, M.E., Russell, GA, Williams, I., Reisner, E.R., & White, RN. (2002). Promoting
Ieaming and school attendance through after-school programs. Student-level
changes in educational performance across TASC’s first three years. Washington,
DC: Policy Studies Associates.

Wolff, T. (2001). A practitioner’s guide to successful coalitions. American Journal of
Community Psychology, 29(2), 173-192.

Wolff, T. (2003). A practice approach to evaluating coalitions. In T. Backer (Ed.),
Evaluating Community Collaborations (pp.57-112). New York: Springer

Publishing.

Wray, R. J., Jupka, K., & Ludwig-Bell, C. (2005, October). A community-wide media
campaign to promote walking in a Missouri town. Preventing Chronic Disease
[Serial Online], 2 (4). Retrieved September, 5, 2008, from
http://www.cdc.gov/pcd/issues/2005/oct/05_0010.htm

 

Yin, R.K., & Kaftarian, SJ. (1997). Introduction: Challenges of community-based

278

program outcome evaluations. Evaluation and Program Planning, 20 (3), 293-
297.

Youniss, J ., McLeIlan, J.A., Su. Y., & Yates, M. (1999). The role of community service
in identity development: Normative, unconventional and deviant orientations.
Journal of Adolescent Research, 14(2), 248-261.

Zakocs, R.C. & Edwards, EM. (2006). What explain community coalition effectiveness?
A review of the literature. (2006). American Journal of Preventive Medicine,
30(4), 351—361.

Zakocs, R.C. & Guckenburg, S. (2007). What coalition factors foster community

capacity? Lessons learned from the Fighting Back Initiative. Health Education
Behavior, 34(2), 354-375.

279

 

    

 
 

G

”1111111111111111][11111111111111115