.a r:— Ea... . . . .. «WE. , a... . . .n. an? , ,. .5. . .i. 7.. a» {L} . . :59... g .4 , . . . . . . . ,. LE. . 2..? .. .2... a... . i... y... . ., u . . $.35... ‘ . . - awwmma . z I.“ 1 as. . t . . _— . . Fm. . :u. 4 . we...“ swans... .. . 7. v. .. .13.... m... . a... .. . cum r... . .. wfifiwfl :flwflflfl , . :3 hr . ..v , . .m........... .3. . , . iffluwmw .3... . . . . . . . . , . , .x. . , . . mu... ”m .1194 5:... 29o“ .. {Una EH. 7. v.7... ..3 .« i... .1 02.1.... .. x .. x \ Mum...“ n , m... 1 1 Ragéwfi... 5...“... .. a. a h ..a. .. t. . :I. 2: 1.5.3 ‘25 4.5... :r 5.... . .. . v A... ..? 2.... = ,P .. .2... 1.? 1:33! . hr . y It a .-\.:...)I...... 53:. k 5.... . b L h» o , . .. . . .a:......x 55¢}... 1 a. . . . .r v . . .I , n .s 1.: .£ I I..{ F .K....:I.:. v .. ..y:A...F.: 7.1m...» . . . ... THESlS 2&1 LIBRARY Michigan State University This is to certify that the dissertation entitled TEACHER PERCEPTIONS AND USE OF PREREFERRAL INTERVENTION PROGRAMS: ROLES, PROCESSES, AND OUTCOMES presented by Amy Pobst has been accepted towards fulfillment of the requirements for Ph.D. degreein School Psychology M r professor Date August 16, 2000 MSU is an Affirmative Action/Equal Opportunity Institution 0-12771 PLACE IN RETURN BOX to remove this checkout from your record. TO AVOID FINES return on or before date due. MAY BE RECALLED with earlier due date if requested. r DATE DUE DATE DUE DATE DUE ’ “330210 200: 11/00 cJCIRcmateoin.pes-p.14 TEACHER PERCEPTIONS AND USE OF PREREFERRAL INTERVENTION PROGRAMS: ROLES, PROCESSES, AND OUTCOMES By Amy Pobst A DISSERTATION Submitted to Michigan State University in partial fulfillment of the requirements for the degree of DOCTOR OF PHILOSOPHY Department of Counseling, Educational Psychology and Special Education 2000 31' ‘ I ..- '9. I .' 'Y'J ‘n. ABSTRACT TEACHER PERCEPTIONS AND USE OF PREREFERRAL INTERVENTION PROGRAMS: ROLES, PROCESSES. AND OUTCOMES By Amy Pobst This study examines teachers’ perceptions of the processes and outcomes of prereferral intervention programs. The purpose of this study is to investigate whether, and to what extent, general education teachers use prereferral intervention programs, and whether use varies along demographic dimensions, such as years of experience, level of education, or grade levels taught, or variables such as satisfaction. This study also explores how teachers, as primary implementers of prereferral interventions, perceive the prereferral intervention processes, and their own roles in problem solving. Finally, we address what teachers perceive their roles to be in implementing interventions designed by the teams, and what their needs are relative to successful implementation of interventions designed through these programs. We accomplished this using complementary methodologies — survey and interview. One hundred fifty four general education elementary school teachers from a midsized urban district completed a questionnaire about their use of and experiences with prereferral interventions programs. Teachers who had never used the teams provided information regarding why they had not used the teams. A subset of thirty teachers who had used the intervention programs were interviewed to obtain more detailed information regarding the processes used by the teams, the outcomes of the problem solving processes, and their perceptions of their roles within these programs. %. , C: = rm ‘f Data indicate the vast majority of teachers in the district use the teams. Those who did not were primarily first year teachers. Teachers’ descriptions of the teams indicated they follow a traditional, hierarchical model of consultation, with teacher involvement generally limited to that of presenting information about the student. The majority of students brought before the prereferral intervention teams were sent on for additional testing to determine whether they might be eligible for special education. Only a few teachers were actually provided with any strategies to use. Their descriptions indicate that the components of problem solving processes recommended in the literature were rarely used. Where there was some adherence to the processes, it was so limited, the interventions that were developed were inadequate in scope and design. Consequently, as the interventions were poor, implementation was weak. Further, teachers were not provided with the requisite conceptual knowledge of the interventions to be able to implement them in a meaningful way. In discussing specific students, teachers were clear they wanted them to be placed in special education. At the same time, teachers wanted professional development that would allow them to do a better job teaching the students. These seeming contradictions were not surprising given the limitations of interventions teachers were provided, which were unlikely to be successful. Teachers’ experiences with the prereferral intervention programs have led them to view the teams as gatekeepers for special education, not as resources for professional development. Results are discussed in the context of expanding the research base on prereferral intervention with an emphasis on determining how to provide teachers with high quality interventions and how to ensure they have adequate resources and supports with which to implement them appropriately. Copyright by AMY POBST 2000 Dedicated to Dick and Nancy Pobst f." -r, I" L: "\' . v ACKNOWLEDGEMENTS I have wanted to be Dr. Pobst since I was seven. My inspiration at that time was Dr. R. Lloyd Pobst, my grandfather. He was a minister, and gave me the key to his office so I could read there instead of fidgeting through lengthy sermons or singing the dreadful “Heavenly Sunshine” song. I remember reading his nametag on his office door, and wanting to be like him. However, I always thought he preached on Sundays and then listened to kids’ hearts with a stethoscope and stuck a tongue depressor down their throats and made them say “Ahh” during the week. So it took a while to figure out what path I’d take on my own. There have been a lot of people since then who have supported my efforts in earning my degree. To all of them, I extend my sincere gratitude and appreciation. In particular, I would like to thank Harvey Clarizio for his support as my doctoral program advisor and committee chair and Walt Hapkiewicz for his encouragement and guidance as a committee member. I would also like to thank Ed Wolfe for his advice on the quantitative analysis, and for providing basic insights and support in a number of areas relative to the design of the study. In particular, I appreciate his sense of humor, and ability to describe Statistics in plain English. I would also like to thank the teachers who took the time to complete the questionnaire. 1 would especially like to thank the thirty teachers who also opted to participate in the interviews and tell me about their experiences. Their enthusiasm and interest in the project and the time they invested in it resulted in my having a much deeper appreciation for their roles, an appreciation I plan to return in thoughtful practice and teaching of my own. vi .-p ~ «'5; l 'l) l 1! .t: Ii. Amy Pobst I am most appreciative and grateful to Linda Patriarca, the director of this dissertation, my mentor, and my friend. She trusted, supported, and encouraged me, and continually critiqued my work with great interest and understanding. She respected what I had to offer, and nurtured what I did not know was there. Without her, this dissertation would not have been possible. My family and friends provided me with every kind of support, from coffee breaks to Bruce Springsteen tickets to trips through the North Woods when I desperately needed breaks from it all. My parents, Dick and Nancy Pobst, my sister-in-law, Judy Pobst, and my friend Bev Wesche were especially constant. Dave and Billie Stewart, Sam, Jake, Becky, and Rachael Pobst, and Joe, Andrea, Tara, and Camilla DelGiudice were also great supports. Garrett Turke, Ron Jones, Sherry Miles, Rosalie Nabors, Brent Myers, Jane Birckhead, Becky Olson, Jim Haines, Annie Varey, Louise Gordon, Erin Wesche, Michael Barnes, and Camilla Wood also deserve special mention. And finally, my grandfather, the original Dr. Pobst, whose special qualities and unconditional love instilled in me when I was seven, the desire to see this through. My thanks to you all. vii TABLE OF CONTENTS LIST OF TABLES .................................................................................................... xii LIST OF FIGURES .................................................................................................. xiv CHAPTER ONE - INTRODUCTION ..................................................................... 1 Statement of the Problem ............................................................................................ 3 Purpose of the Study ................................................................................................... 4 Theoretical Perspectives .............................................................................................. 5 Behavioral Consultation ....................................................................................... 5 Collaborative Consultation ................................................................................... 7 Collaboration ....................................................................................................... 9 Overview of the Study ................................................................................................ I 1 CHAPTER TWO - REVIEW OF THE LITERATURE ......................................... 14 Description of the Research Base ................................................................................ l4 Participants and Roles ................................................................................................. 15 Team Membership ............................................................................................... 15 Teachers’ Roles .................................................................................................... 16 Team Use ............................................................................................................. 17 Teacher Perceptions ............................................................................................. 18 Problem Solving Procedures and Communication ....................................................... 19 Problem Solving ................................................................................................... 19 Communication .................................................................................................... 2] Intervention Implementation ....................................................................................... 22 Types of Interventions .......................................................................................... 23 Intervention Implementation ................................................................................ 24 Teachers’ Perceptions of Interventions ................................................................. 24 Justification for the Study ............................................................................................ 25 Chapter Summary and Limitations of Previous Studies ............................................... 27 CHAPTER THREE - METHODOLOGY .............................................................. 29 Instrument Development ............................................................................................. 29 Questionnaire Development ................................................................................. 30 Pilot Testing the Questionnaire ............................................................................. 33 Interview Development ........................................................................................ 34 Interview Development - Former Users ............................................................... 35 Pilot Testing the Former Teachers’ Interview ....................................................... 37 Interview Development — Recent Users ................................................................ 37 Pilot Testing the Recent Users’ Interview ............................................................. 4O viii Informed Consent ................................................................................................. 4O Reliability. .................................................................................................................. 41 Validity. . .. .................................................................................................................. 42 Characteristics of the Sample ...................................................................................... 44 Setting .................................................................................................................. 44 Sampling Methodology ........................................................................................ 45 Data Collection Procedures ......................................................................................... 46 Distribution and Response Rates for Questionnaires ............................................. 47 Presentations at staff meetings ......................................................................... 47 Building representatives .................................................................................. 48 Building principal ............................................................................................ 48 Follow Up Study of Nonresponders ...................................................................... 49 Data Collection - Interviews ................................................................................ 50 Response Rates - Interviews ................................................................................ 50 Follow Up Study — Questionnaires ....................................................................... 51 Research Design ......................................................................................................... 53 Data Analysis .............................................................................................................. 54 Research Questions ..................................................................................................... 55 Question #1 .......................................................................................................... 55 Question #2 .......................................................................................................... 56 Question #3 .......................................................................................................... 58 Question #4 .......................................................................................................... 59 Additional Analyses — Satisfaction ....................................................................... 60 Limitations .................................................................................................................. 60 Chapter Summary ....................................................................................................... 61 CHAPTER FOUR — RESULTS ............................................................................... 63 Question One: Team Use ............................................................................................ 63 Characteristics of the Sample ............................................................................... 63 Subgroup Analyses .............................................................................................. 65 Subgroup l - Never Users and Users ............................................................... 66 Subgroup 2 — Former Users and Recent Users ................................................. 66 Subgroup 3 — Users Interviewed and Users Not Interviewed ............................ 67 Results of the Subgroup Analyses ........................................................................ 68 Subgroup l — Never Users and Users ............................................................... 68 Subgroup 2 — Former Users and Recent Users ................................................. 71 Subgroup 3 — Users Interviewed and Users Not Interviewed ............................ 71 Additional Analyses — Nonresponders and Responders .................................... 71 Question One: Summary ...................................................................................... 73 Characteristics of Referrals ......................................................................................... 73 Team Membership and Referral Procedures ................................................................ 74 Team Membership ............................................................................................... 74 Length of the Meeting .......................................................................................... 76 Expectations ......................................................................................................... 76 Question Two: Problem Solving and Communication ................................................. 77 Problem Identification and Clarification ............................................................... 78 Types of information gathered ......................................................................... 79 Types of information reviewed at the meeting ................................................. 79 Problem identification — summary ................................................................... 82 Plan Development ................................................................................................ 83 Components of the Process .............................................................................. 83 Plan Development ........................................................................................... 84 Teachers’ roles in plan development .......................................................... 86 Plan development — summary .................................................................... 89 Agreement with the Plan ...................................................................................... 89 Procedures for Evaluation .................................................................................... 91 Question Two: Summary ...................................................................................... 91 Involvement and Demographic Patterns ...................................................................... 92 Question Three: Intervention Implementation ............................................................. 95 Responsibility for Intervention Implementation .................................................... 96 Implementation of Strategies ................................................................................ 97 Teachers’ Perceptions of Strategies ...................................................................... 98 Question Three: Summary .................................................................................... 99 Goals of Prereferral Intervention Revisited ................................................................ 100 Supports and Resources ............................................................................................ 101 Time ................................................................................................................ 101 Time to the referral ........................................................................................ 102 Time between the referral and meeting .......................................................... 102 Time at the initial meeting ............................................................................. 103 Time team members spent in the building ...................................................... 103 Training and Service Delivery ............................................................................ 105 Training in team procedures .......................................................................... 105 Training for first year teachers ....................................................................... 108 Instruction in teaching difficult to teach students ........................................... 108 Service Delivery ................................................................................................. 109 Observation of teachers and student performance .......................................... 109 In-class modeling, consultation and support ................................................... 109 Specific instruction in how to modify assignments and adapt materials .......... 110 More Voice in Decision Making ........................................................................ 11 1 Question Four: Summary ................................................................................... 113 Additional Analyses — Teacher Satisfaction ............................................................... 114 CHAPTER FIVE - DISCUSSION ......................................................................... 1 16 Summary of the Study ............................................................................................... 116 Limitations ................................................................................................................ 1 19 Discussion and Conclusions ...................................................................................... 121 Problem Solving Processes ................................................................................. 122 Components of the Problem Solving Process ................................................. 122 Length of the Meeting ................................................................................... 123 Intervention Implementation .......................................................................... 124 Supports and Resources ................................................................................. 125 Teachers’ Roles and Participation ...................................................................... 126 In an. R\PL 8“... ath\nl.UELFC~ First Year Teachers ........................................................................................ 126 Team Membership ......................................................................................... 127 Professional Development ............................................................................. 128 Team Training ............................................................................................... 130 Characteristics of Referrals: Gender, Retention, and Referral History ........................ 131 Closing Comments: Reflections on the Current Study ............................................... 133 APPENDICES ......................................................................................................... 136 A Questionnaire ..................................................................................................... 137 B Interview for Former Users ................................................................................ 144 C Interview for Recent Users ................................................................................. 147 D Information and Consent for Questionnaire ........................................................ 152 E Informed Consent for Interview ......................................................................... 154 F Indication of Willingness to Consider Participation in Interview ........................ 156 G Characteristics of Referrals ................................................................................ 158 Numbers of Students Referred ....................................................................... 159 Types of Problems Referred .......................................................................... 160 Gender, Retention, and Referral Histories ...................................................... 160 Prior to Referral ............................................................................................. 161 Summary ....................................................................................................... 164 LIST OF REFERENCES ....................................................................................... 166 10 ll 12 I3 14 15 16 I7 18 I9 20 LIST OF TABLES Components of the Problem Solving Model ......................................................... 20 Questions and Response Choices for Teachers who had Never Used the Teams 31 Questionnaire Items — Distributed to All Teachers ................................................ 33 Response Choices for Teachers who had not Used the Teams Within the Previous 18 Months ......................................................................................... 35 Items Included in the Interview for “Former Users” ............................................. 36 Items Included in the Interview for “Recent Users” .............................................. 39 Questionnaire Distribution and Return Rates per Type of Distribution Method 49 Questionnaire Distribution and Return Rates — Initial and Follow Up Distributions .................................................................................................... 52 Distribution of Teachers in the District According to Grade Taught (Questionnaire and Directory Data) ................................................................. 53 Questionnaire Items Addressing Teacher Use of Teams ....................................... 56 Questionnaire and Interview Items Addressing Problem Solving and Communication ............................................................................................... 57 Questionnaire Variables Used to Address Correlates of Involvement ................... 58 Questionnaire and Interview Items Addressing Intervention Implementation ........ 59 Questionnaire and Interview Items Addressing Supports and Resources Needed .. 59 Questionnaire and Interview Items Addressing Satisfaction .................................. 60 Characteristics of the Sample ............................................................................... 65 Questionnaire Items Used for analysis of Subgroup Differences: Users Interviewed and Not Interviewed ..................................................................... 68 Reasons Given by Teachers for Never Having Used the Teams ............................ 69 Reasons Given for Never Having Used Teams — First Year Teachers Excluded 70 Comparisons of Responses of Teachers Interviewed and Not Interviewed ............ 71 xii 21 22 23 24 25 26 27 28 29 30 31 32 33 Teacher Reports of Persons in Attendance at Prereferral Meetings ....................... 75 Teacher Reports of Types and Sources of Information Reviewed at Meetings ...... 80 Teacher Reports of Use of Components of Plan Development .............................. 84 Plans Developed in the Prereferral Process ........................................................... 85 Teacher Reports of Who was Most Responsible for Developing Interventions ..... 86 Teacher Reports of their Own Role in the Development of Interventions .............. 88 Teacher Reports of their Team Relative to Collaboration or Expert Models .......... 88 Teacher Reports of Agreement with the Plan ........................................................ 90 Teacher Reports of Involvement in Plan Development ......................................... 93 Contribution of Demographics to Involvement ..................................................... 94 Interaction Between Years of Teaching and Involvement ..................................... 95 Strategies Recommended by Teams and Implemented by Teachers ...................... 98 Teacher Ratings of Satisfaction ...... . ................................................................... 115 6-1 Number of Students Referred to Teams - Current and Previous School Year ..... 159 G-2 Types of Problems Referred ............................................................................... 160 6-3 Gender, Retention and Referral History of Students Referred ............................. 161 G-4 Number of Interventions Teachers Reported Implementing Prior to Refen'ing Students ........................................................................................................ 162 GS Strategies Teachers Used to Address Problems before Referring Students .......... 163 xiii O- (J‘- Prereferra Cons. ' Prereferra C on)“: Prerefcrra Modes 0.: —o . , 133171.316? 0“»- ”Icncnii; ,l Rmaduny. R631,» _L \A.\. iP'D"1~.l “\I \,’~ LIST OF FIGURES Prereferral Intervention Programs and Schema Associated with Behavioral Consultation .................................................................................................... 7 Prereferral Intervention Programs and Schema Associated with Collaborative Consultation .................................................................................................... 9 Prereferral Intervention Programs and Schema Associated with Collaboration ..... 11 Models of Consultation and Associated Prereferral Intervention Programs ........... 14 Instruments Developed and Groups to be Administered Each Instrument ............. 30 Differentiation between Team Users and Instruments Developed ......................... 35 Relationship to Respondents of Subgroup 1: Never Users and Users .................... 66 Relationship to Respondents of Subgroup 2: Former Users and Recent Users ...... 67 Relationship to Respondents of Subgroup 3: Users Interviewed and Not Interviewed ..................................................................................................... 68 xiv T n- TIQ t. l .~-~ 89.671. ‘uui. U hips-23:10:: C r bacigraunis c 1513 188‘). Br lessor. 1749? are :rtpacted. laiztzdgais it CHAPTER ONE - INTRODUCTION The challenges classroom teachers confront are becoming more complex as general education attempts to integrate the needs of an increasingly diverse student population. Children from a variety of family structures, cultures, ethnic and economic backgrounds come to school with differing levels of preparation for traditional school tasks (Bay, Bryan, & O'Connor, 1994; Knofi", Hines, & Kromrey, 1995a; Pugach & Johnson, 1995). As diversity in these areas increases, school readiness and performance are impacted, adding to teachers’ responsibilities. To further complicate matters, the Individuals with Disabilities in Education Act (IDEA, 1997) mandates that students with disabilities be educated in the Least Restrictive Environment, and because of this, more students with disabilities are being taught in general education. Often, however, teachers do not perceive themselves to be prepared to meet these added demands (Coben, Thomas, Sattler, & Morsink, 1997; Schumm, Vaughn, Gordon, & Rothlein, 1997). To assist them in meeting student needs, support is provided to teachers through several different venues, including a variety of consultant services, inservice programming, and collaborative efforts such as team teaching and co-teaching (Idol, 1993). One popular way of providing support to teachers is through consultation between general education and support stafi’, often through a multidisciplinary team approach known as prereferral intervention. Typically, this involves teams of specialists and generalists representing the fields of special education and general education working together to help students who are “difficult to teach” succeed in general education (Bahr, Fuchs, Fuchs, & Fernstrom, 1993; Kovaleski, Tucker, & Stevens, 1996). Initiated immediately at the teachers’ identification of a problem (Fuchs, Fuchs, Bahr, Fernstrom, 6; Stacker. 10* approach to d: reg-.13: educa: 19%. Pugam l 94:33.. so 51;; K‘JQZIOR. a‘\ \ & Stecker, 1990c), prereferral procedures generally incorporate a multidisciplinary team approach to designing systematic interventions for implementation by the teacher in the regular education classroom (Graden, Casey, & Christenson, 1985b; Kovaleski et al., 1996; Pugach & Johnson, 1989). These interventions are intended to provide classroom support so students’ problems do not become severe enough to warrant referral to special education, avoiding the problems of stigmatization and segregation associated with labeling students. Further, professionals view prereferral intervention as a usefirl tool in providing teachers with ongoing and targeted professional development, enhancing teachers’ skills so future problems might be reduced (Bahr, 1994; Murdick & Petch- Hogan, 1996; Nelson, Smith, Taylor, Dodd, & Reavis, 1992). Prereferral programs have been extensively researched. Studies have looked at factors such as team membership (Bahr, 1994; Carter & Sugai, 1989), types of problems referred to teams (Chalfant, Pysh, Miros, Bradshaw, & Adams, 1991; Chalfant & Pysh, 1989; Johnson & Pugach, 1991; Korinek & McLaughlin, 1996; Whitfield, 1996), and processes followed in intervention development (Chalfant, Pysh, & Moultrie, I979; Fuchs et al., 1990a; Graden, 1989; Hartman & Fay, 1996; Kovaleski et al., 1996; Pugach & Johnson, 1988; Pugach & Johnson, 1995; Rosenfield & Gravois, 1996). Research has also examined areas such as consumer satisfaction (Bay et al., 1994; Chalfant et al., 1991; Chalfant & Pysh, 1989; Fuchs & Fuchs, 1989; Graden, 1989; Nelson, 1991), and impact of prereferral programs on rates of referral to special education (Chalfant et al., 1979; Graden, Casey, & Bonstrom, 1985a; Hartman & Fay, 1996; Kovaleski et al., 1996; Pugach & Johnson, 1995; Rosenfield, 1992; Short & Talley, 1996). Prereferral intervention programs involve teams of specialists developing interventions for teachers s 1' tomement . if 383.11% inter ‘ \ ‘22:. df’t €113? . rep-tn Mm adeqmteft in; “1sz16 0L - m'STt .3 JL' CF"? “‘v- ., i I ‘:“:"1 .':O.rh , ll ‘ k\n if t 4" r ulfig to implement. The use of specific problem solving procedures by these teams should lead to quality interventions that meet student needs. However, research indicates specialists often develop or recommend prereferral interventions that are limited in quality or in depth (Flugum & Reschly, 1994; Pobst & Patriarca, 1999), and teachers often fail to adequately implement interventions (Noell & Witt, 1999). Statement of the Problem As the implementation of interventions developed by the prereferral intervention teams relies almost exclusively on the classroom teacher, it seems logical these teachers have pivotal roles on the teams. Despite this, there is very little research about the teacher’s involvement’on these teams and their views of prereferral programs. For example, there is an emphasis in the literature on making consultation “collaborative,” and engaging teachers more fully in prereferral processes and procedures. However, the Zeitgeist of consultation research implies that teachers stand in the way of intervention implementation and, therefore, create or maintain problems for students. Phrases such as “teacher resistance,” and “teacher failure to adhere to treatment plans,” are not uncommon in the literature. These suggest teachers are regarded as obstacles, either impeding progress by failing to follow through with recommendations, or culpable when interventions are unsuccessful (Mortenson & Witt, 1998; Noel] & Witt, 1999). If prereferral intervention programs are to be successful, teachers must implement the interventions that are developed. 1f specialists such as school psychologists are to be successful in their roles as consultants, they need to engage teachers’ productive participation on teams and support teachers in their efforts to implement the recommended interventions. At the same time, psychologists and other team members 11111315313558 : CBSSIC‘OIT. 3.51:. IQ flex trailers. at n: ‘ i~ ’. - “if.” pity-5:57: Kim their '9. _. VE‘V‘H ~' _ ‘ ' uk’lr i' ' Vs , 'va ‘ .. A, bétmji. \.N.'T must balance the need for targeted, specific interventions with the realities of a typical classroom and its constraints and competing demands. To develop prereferral intervention programs that effectively meet the needs of teachers, we need more data regarding teachers‘ experiences, both positive and negative, with prereferral intervention teams. We need more information about how teachers perceive their roles within the contexts of team firnctioning, and what their needs are in terms of intervention development and implementation. We also need to determine what kinds of support teachers need to make more complex and integrated interventions work. Purpose of the Study This study is designed to contribute to the understanding of the functioning of prereferral intervention teams. It examines how teachers, as primary implementers of prereferral interventions, perceive the prereferral intervention processes and how they perceive their own roles in the design and implementation of interventions. This study addresses whether, and to what extent, teachers use prereferral intervention programs, and to what extent their perceptions influence their use of these teams. Finally, it examines what teachers perceive their needs to be relative to intervention programs, and how they perceive these programs could be improved to better facilitate intervention implementation. Within this context, goals are to inform consultants, such as school psychologists, of factors that need to be considered when working with teachers on prereferral intervention teams. To us: _'{',- H. w pennant. t mous were 1 Fret-cf“; mammal ix: r Y “‘2‘ - - ~e4cécfil\-~ p _.h (7' l senices for :3: 1'2; use of in ' .-f . _‘ . -~R‘. ' o “03 115 I it-) ‘6. -\i;& Theoretical Perspectives To understand prereferral intervention generally, and the nature of this study specifically, one must be acquainted with the theoretical perspectives that undergird the various prereferral programs cited in the literature. Prereferral intervention programs fall under the domain of consultation in educational literature. Consultation is defined as an interactive problem-solving relationship between a consultant (the specialist) and a consultee (the teacher), regarding services for the benefit of a client (the student) (Tharp, 1975). It is predicated on the idea that use of indirect services facilitates efficient and effective use of limited consultant resources, and that, minimally, both the consultee and the client benefit from the relationship with the consultant. There are three major models of consultation at the forefront of school consultation literature: Behavioral Consultation, Collaborative Consultation, and Collaboration. Although the models have a lot in common, they are also quite different, most notably in the theories that support them and the way in which each conceptualizes the relationship between the consultant and consultee. In this section, we describe these three models relative to underlying theories, how problems are conceptualized, and the relationship between the consultant and consultee. Behavioral Consultation Probably the most prominent model that guides consultation in the schools is Behavioral Consultation. Grounded in social learning theory and applied behavior analysis, its basic assumptions are that environmental manipulation will change client behavior (Conoley & Conoley, 1982; Idol, 1993; Sugai & Tindal, 1993). Behaviors are conssdered it be learned uk- 0. :1". cons-aft. an: mama '. the heir-- IN 2: a . J. . - fake-ye ”€35, considered fitnctionally related to factors within the immediate environment. Student problems are conceptualized from a learning perspective; that is, problematic or deficient skills stem from idiosyncratic reinforcement histories. Different or desired behaviors can be learned when appropriate feedback is received (Conoley & Conoley, 1982). The task of the consultant is to define the problem, isolate the environmental variables that elicit and maintain the target behavior, and design interventions that reduce the likelihood that the behavior will continue. Additionally, the consultant devises interventions that can produce new, desired behaviors. Behavioral Consultation is a hierarchical model that assumes the consultant knows something the consultee does not. The focus of consultation is for the consultant to impart that knowledge to the consultee. This model emphasizes the use of the specialized training or expertise of the consultant in developing interventions. Through these processes, the consultant increases the consultee’s specific skills and knowledge so the consultee can solve the client’s problems (Sugai & Tindal, 1993). The task of the consultee, or in this case, the teacher, is to merely implement interventions designed by others. The prereferral intervention programs most closely aligned with this model and the relationship between the consultant; consultee and client are in Figure 1 below. ASSOCIATED PREREFERRAL INTERVENTION PROGRAMS: BEHAVIORAL CONSULTATION | Instructional Consultation Teams (Rosenfield & Gravois. 1996) l Mainstream Assistance Teams (Fuchs. 1987) CONSULTANT (Specialist) 1 l i CONSULTEE (Teacher) 1 l l CLIENT (Student) Figure 1 Prereferral Intervention Programs and Schema Assomted with Behavioral Consultation Collaborative Consultation Although the Behavioral Consultation model described in the previous section dominates the school consultation field, programs that use this approach are frequently criticized for minimizing, or trivializing, the role of the teacher in problem solving (Idol, Paolucci-Whitcomb, & Nevin, 1986). Attempts to reconcile problems with failures in intervention implementation have resulted in a call for programs that promote teacher ownership for the interventions that are designed. Principles of organizational psychology have contributed to the development of the second problem-solving model addressed here, Collaborative Consultation (Conoley & Conoley, 1982). "it, stare: . swam: interactions. Err}. desred 1 icamg next 11’. team men Efrain-3:1. ( . a 'H? .p. y‘_'. “hfiflwa IUa-l § an... . fl. “1315 or -'. I ~ a. {9.1.5. \‘ “I" t 9:5 on. ‘ ~ LI“ ..1 \t O "E ,, " c Collaborative Consultation is based on principles of collaboration that emphasize parity, shared power, and decision-making. The relationships between the individuals involved in the processes are considered critical, as are the understanding of roles and interactions, and an awareness of interpersonal interactions that contribute to outcomes, both desired and undesired. One basic assumption of this approach is that planning, learning new behavior, and adapting new routines will, at some point, involve and affect all team members (West & Idol, 1987), and as such, benefit all team members. As with Behavioral Consultation, Collaborative Consultation involves databased decisions made through functional analysis of problem behaviors. However, there is a particular emphasis on the role of positive reinforcement, not only in the interventions developed for the consultee to use with the client, but also in the relationship between the consultant and the consultee. While responsibility for implementing the interventions is largely the role of the consultee, there is an emphasis on identifying and respecting the teacher as a co-equal. Each person involved in the consultation plays an active role in the design and evaluation of the program (Graden, 1989; Heron & Harris, 1987; Idol, 1993; Idol et al., 1986; Jordan, 1994). The prereferral intervention programs most closely aligned with this model and the relationship between the consultant; consultee and client are in Figure 2. COLLABORATIVE-CONSULTATION | Teacher Assistance Teams (Chalfant et al.. 1979) | Prereferral Intervention Senice Delivery Systems (Graden et al.. 1985b) I Instructional Support Teams (Kovaleski et al.. 1996) l Intervention Assistance Teams (Ohio State Department of Education. 1985) CONSULTANT _) _’ _)_’ CONSULTEE (Specialist) 6—) —> (Teacher) i l l CLIENT (Student) Figure 2 Prerefe‘rral Intervention Programs and Schema Assgiated with Collaborative Consultation Collaboration The third and final model of consultation addressed here, Collaboration, has evolved from constructivist theory. The model includes critical components of constructivist theory, identified by Nyikos and Hashimoto (1997) as involving: self- regulation, use of language as a mediation tool, Zone of Proximal Development, problem solving, scaffolding, and application of critical thinking skills. Three basic assumptions underlie Collaboration. First, the consultee is understood to have the knowledge necessary to resolve the problems, but requires the time and structure to create new responses to problems. Second, using strategic and active thinking skills allows the consultee to readily identify problems and construct solutions. Third, having the support . 1 ‘ 3k. reia'tik‘: V‘ . b ‘1 .>‘~§ - ""'~.' r' d P ' bh_,\b ‘. ...\ of a colleague enhances the consultee’s ability to be flexible in problem solving. Finally, the relationship between participants must be reciprocal, promoting empowerment through shared responsibility (Pugach & Johnson, 1995). The only prereferral intervention program we found that truly exemplifies the characteristics of Constructivism is Peer Collaboration (Pugach & Johnson, 1988), depicted in Figure 3. The learning or behavior problems exhibited by clients (students) are conceptualized as the product of an interaction between teacher characteristics and student characteristics, both positive and negative. The purpose of the problem solving process is to help the teacher a) develop a clearer understanding of the problems, and b) to consider ways in which changing teacher behaviors and teaching routines can result in successful student performance. The role of the consultee, or initiator (the teacher bringing the problem is called the “initiator”) is to solve the problem. The consultant (called the “facilitator”) simply ensures the processes are used appropriately by the initiator, in ways that will result in the teacher arriving at a solution (i.e. clarifying questions are exhausted and the problem is clearly defined). A critical component of this model is that it is reciprocal. It is anticipated that over the course of a partnership, each teacher will encounter difficulties that can be addressed in this manner, such that each will function as either initiator or facilitator. IO (I . r {v-92 ‘ F 5' .. '1‘.-. 8| si‘.“: “X 3735.113 (3-. C ‘Vr‘ :L’\~u‘ar e y. W 1 Peer Collaboration (Pugach & Johnson. 1988) CONSULTANT-F AC lLITATOR CONSULTANT-FACILITATOR CONSULTEE-INITIATOR H CONSULTEE-INITIATOR (Teacher) 9 (Teacher) Figure 3 Prereferral Intervention Programs and Schemflssociated with Collaboration It is logical to assume each of these theoretical perspectives guiding prereferral programs could impact differentially on teachers’ perceptions and use of programs. In particular, the results of research on consultation models indicate teachers express preferences for collaborative rather than hierarchical interactions (West & Idol, 1987). It is beyond the scope of this study to control for the type of consultation used by individual teams. However, in this study, we will address whether teachers consider their experiences collaborative or hierarchical and examine their responses in light of these theoretical perspectives. Overview of the Study This study involves two complementary methodologies — survey and interview. Approximately 200 general education elementary school teachers from a midsize urban district were surveyed regarding their use of and experiences with prereferral intervention programs. A subset of thirty teachers who had used the intervention programs were interviewed to obtain more detailed information regarding their satisfaction with their roles, the processes used by the teams, and the outcomes of the interventions. Teachers who had not used the teams were asked to provide information regarding why they had 11 chosen not 1.‘ mission tested to \ea ‘36.; :ncreasel and hf“ flit“: scariest: 1c 'Mw ; \a a. t\§.¢ti ‘H‘1‘sl‘ “~"“ I! ~- “P“M‘finluc “91’ in». .— r‘lf\> a]. . 'J >‘ O a .L _ ..scwc chosen not to use the teams. Data are analyzed to identify correlates of team use and satisfaction with teams, and to identify recurring themes, patterns, and critical issues related to team firnctioning, problem solving and collaboration. Data from this study will help increase understanding of teachers’ perceptions of prereferral intervention programs and how these perceptions impact on teachers’ use of these teams. The data will also contribute to understanding how teachers who use prereferral intervention teams perceive team firnctioning relative to their own roles, to problem solving processes and communication, and the implementation of interventions. Data will also provide us with teachers’ perceptions of what is needed to make prereferral intervention programs successful. The overarching questions that guide this study are: What are teachers’ perceptions of prereferral intervention programs and what factors influence their decision to use the teams? The specific research questions are: 1. Who uses prereferral intervention teams? 2. What are teachers’ perceptions of the problem solving processes of prereferral intervention teams and of the communication that facilitates problem solving? 3. What are teachers’ perceptions of their own roles on these teams relative to their implementation of prereferral interventions in the classroom? 4. What kinds of supports and resources do teachers say they need in order to make prereferral interventions succeed? There are four remaining chapters in this dissertation. Chapter Two contains the review of literature on prereferral intervention and the justification for the present study. 12 fhzgter Thre commas the ' 1 . l . v.1. .17.; a Chapter Three focuses on the qualitative and quantitative methodology. Chapter Four contains the results of the analyses and Chapter Five provides a discussion of the results with implications for practice and future research. 13 Thrs : teachers pet. an: 3161? I033 'I rinse-r. »\'.. ,-'~‘~m .31“ CHAPTER TWO - REVIEW OF THE LITERATURE This review of the literature addresses three areas relevant to understanding teachers’ perceptions and use of prereferral intervention programs: 1) team participants and their roles, 2) procedures and communication patterns that facilitate successfiil problem solving and intervention development, and 3) intervention implementation. Description of the Research Base Before proceeding, it may be helpful to the reader to know that a large portion of the review of the literature covers research related to seven specific prereferral intervention programs, organized according to the theoretical models discussed in Chapter One, with which they are most closely aligned (See Figure 4 below). BEHAVIORAL COLLABORATIVE- CONSULTATION CONSULTATION W | | 1 Instructional Consultation Teams Teacher Assistance Teams Peer Collaboration (Rosenfield & Gravois. 1996) (Chalfant et al.. 1979) (Pugach & Johnson. 1988) l l Mainstream Assistance Teams Prerefenal Intervention (Fuchs. 1987) Service Delivery Systems (Graden et al.. 1985b) l Instructional Support Teams (Kovaleski et al.. 1996) | Intervention Assistance Teams (Ohio State Department of Education 1985) F igure 4 Models of Consultation and Associated Prereferral Intervention Programs Research from five of these programs (Instructional Consultation Teams, Mainstream Assistance Teams, Teacher Assistance Teams, Prereferral Intervention Service Delivery Systems, and Peer Collaboration) generally represent University-based research projects. The other two are statewide initiatives, derivatives of earlier programs, mandated and prescribed at the state level (Instructional Support Teams in Pennsylvania 14 l and Intenerrk adiressed II‘. In :11; questing in: '1) I: 313,31“ Winter‘s teachers it: Iran: \lcr and Intervention Assistance Teams in Ohio). Beyond these, the remainder of the research addressed in this literature review covers less clearly defined programs. Participants and Roles In this section, we discuss what research tells us relative to our first two questions: “Who uses prereferral intervention teams?” First, we will describe what is known about team composition, the role of the teacher on the team, and teachers’ perceptions of their roles on these teams. We will then discuss what is known about the teachers who use the teams and those who do not. Team Membership A lot of what is known about the composition of prereferral intervention programs comes from a survey of state departments of education conducted by Carter and Sugai (1989). Results of this survey, which included responses from 49 states, indicate teams are likely to be multidisciplinary, and to include the referring teacher, principal, school psychologist, and a special education teacher or consultant. Within states, reports from program evaluators in Pennsylvania (Kovaleski et al., 1996) and Ohio (Ohio State Department of Education, 1985), and surveys of directors of special education in Michigan (Bahr, 1994), and of elementary schools in Illinois (Whitten & Dicker, 1995) inform us program membership largely follows these patterns. However, we also know group composition varies. In some instances, the groups involve the referring teacher meeting with a school psychologist or special education consultant (Fuchs et al., 1990c; Graden et al., 1985b). In some programs, the referring teacher meets with small groups of teachers (Chalfant et al., 1979), in others, pairs of teachers are trained to work together (Pugach & Johnson, 1988). In yet other programs, the teacher is involved with 15 ‘ Ann-, ,I' m kkstxna 3:173 \ B‘u: , .. $~“.Il.‘-‘ ”T's-.4 “m,_\‘ fib’ " -\ i ' \Q. multidisciplinary groups that may have as many as fourteen permanent members (Kovaleski et al., 1996; Rosenfield & Gravois, 1996). We know the referring teacher is a member of all these teams in some capacity. Teachers’ Roles The literature on prereferral intervention stresses the importance of collaboration in consultation. However, it also appears that the majority of prereferral intervention programs continue to adhere to traditional and hierarchical roles. The role of the specialist or consultant is to define the problem, and develop and evaluate the interventions. The role of the teacher is to implement the interventions. The most conspicuous example of this is the Instructional Consultation Team, where the teacher is not even included in team meetings, but rather, is represented during all planning stages, by a liaison (Rosenfield & Gravois, 1996). Conversely, the most notable exception to the traditional consultant-consultee roles is Peer Collaboration. Here, the referring teacher is trained in the problem solving processes. The referring teacher has primary responsibility for calling the meeting, defining the problem, and developing and implementing interventions (Johnson & Pugach, 1991; Pugach & Johnson, 1988; Pugach & Johnson, 1995). However, in the majority of the studies reviewed, the role of the referring teacher is somewhere between these two extremes. Typically, the referring teacher provides information, usually through an interview or a written referral, and attends a meeting where interventions are developed (Chalfant et al., 1979; Fuchs, Fuchs, & Bahr, 1990b; Graden et al., 1985b; Kovaleski et al., 1996; Ohio State Department of Education, 1985; Vaughn, Schumm, Jallad, Slusher, & Saumell, 1996). We know teachers are involved in the development of interventions in some capacity, but we are not really sure what that 16 imohemem mmdmgmi “hi limited. ac 1 mermwn mnmsmn tameccn involvement looks like. Most typically, it appears the teacher’s role is a passive one, providing information to the consultants who then develop the interventions. While information about the teachers’ role in the design of interventions is limited, we know teachers have the primary responsibility for implementing the interventions developed (Bahr, 1994). The Carter and Sugai (1989), and Bahr (1994) surveys reveal that teachers sometimes receive help from paraprofessionals or fi'om teacher consultants. For example, Instructional Support Teams allow the teacher consultant to work directly in the classroom with the student and teacher for up to 60 days in order to support intervention implementation, as well as gather outcome data and make program modifications (Kovaleski et al., 1996). However, we do not know whether, or how often, teachers need support in implementing interventions, or whether they actually receive that support, or if they do, whether the support is adequate. Moreover, we do not know whether teachers perceive themselves as able to implement the recommendations, or whether they consider the interventions to be realistic given their other responsibilities. Team Use While the data are consistent in informing us that teams include teachers, we were unable to find data about which teachers use prereferral programs and which teachers do not use them. We have some demographic data available in the form of descriptions of sample populations in several studies of prereferral intervention programs (Chalfant et al., 1991; Chalfant & Pysh, 1989; Fuchs, 1987; Graden et al., 1985a; Johnson & Pugach, 1991; Rosenfield, 1992; Short & Talley, 1996). Unfortunately, these were not sufficiently detailed for us to be able to reach any conclusions about who uses the programs. The 17 ataxlabe re.“ teasers xx 2 "011211 b» fl \v With 53'. the} do net 1 1115! 51:11 .l W 81.”..le {Q m. ~ -nue \OiU\ P... . v ~_‘. .11 dds-‘3.“ l .1,‘ t Inth‘ available research on teacher use of consultation offers mixed results. Some research says teachers with less than six years experience are more likely to engage in consultation, probably because they perceive themselves as needing more support (Mann, 1973). Other research says teachers with more experience are more receptive to consultation, because they do not see asking for help as being indicative of poor teaching skill (Kahl & Fine, 1978). Still other research says teachers who use consultation are those who have access to skilled consultants (Kutsick, Gutkin, & Witt, 1991; Stenger, Tollefson, & Fine, 1992). Additionally, we do not know whether participation in these programs is driven by its being prerequisite to referral to special education. Thus, essentially, we do not know whether prereferral teams are used by novice teachers, or by experienced teachers, or by some combination thereof. Finally, and of equal importance, we have no information about the teachers who do not use these teams. Teacher Perceptions Finally, we were unable to find any information, either from literature specific to prereferral intervention programs or fi'om consultation in general, related to teachers’ Perceptions of their roles in prereferral intervention programs. We were also unable to find any information about the perceptions of these programs by teachers who do not use Prereferral intervention programs. Without these data, we cannot reach conclusions about What either group perceives as necessary to improve team firnctioning. At this point, let us summarize what we know relative to our first question: “Who uSes prereferral intervention teams?” We know the majority of prereferral programs have Participants from multiple disciplines. The programs are typically based on expert-to- teacher consultation, with the consultant bearing responsibility for problem definition and I8 mtenentlor. rape “S‘ih ‘tll ‘|‘\ the amount '. “11116116 1; no. resear 1': he“ file“ I DO resear: 1.181%me intervention development. In these programs, the classroom teacher has primary responsibility for implementing the interventions. Differences among programs relate to the amount and type of support the teacher receives for implementing interventions. While we know there is a broad emphasis on teacher ownership and ensuring the teacher feels a part of the team, we do not know whether this, in fact, occurs. Further, we found no research about teachers’ perceptions of their roles on prereferral intervention teams, or how their perceptions of their roles influence their use of the programs. Finally, we found no research about teachers who do not use prereferral intervention teams or their perceptions of these teams. Problem Solving Procedures and Communication The literature states repeatedly that specific problem solving procedures and effective communication are critical to problem solving by prereferral intervention teams (Adelman & Taylor, 1998; Bruskewitz, 1998; Idol, 1997). In this section, we examine the literature related to our second question “What are teachers’ perceptions of the problem solving processes of prereferral intervention teams and of the communication that facilitates problem solving?” Here we discuss what is known about the procedures and communication patterns that best facilitate problem solving. We then discuss what is known about teachers’ perceptions of these problem-solving procedures and communications. Finally, we address what is known about how these perceptions impact teachers’ use of teams. Problem Solving Regardless of their theoretical orientation, all of the prereferral intervention programs we found in the literature advocate the use of a problem solving procedure 19 dfi'llfd tit. 10111 10110“ | l dexelopmet mct‘ltfscatz. these indm €157.38! 1‘: K3‘~11€§t;1 e ELSE R0 if. derived from applied behavior analysis. This procedure involves some variations of the four following steps: 1) Problem identification and clarification, 2) plan and intervention development, 3) intervention implementation, and 4) evaluation of interventions and modification as necessary (Sugai & Tindal, 1993; West & Idol, 1987). Shown in Table 1, these indicants or their parallels are considered crucial components of problem solving in exemplar programs (Chalfant et al., 1979; Fuchs et al., 1990d; Graden et al., 1985a; Kovaleski et al., 1996; Ohio State Department of Education, 1985; Pugach & Johnson, 1988; Rosenfield & Gravois, 1996). Table 1 Components of the Problem Solving Model Problem Identification and Clarification Development of Plans and Interventions Implementation of Interventions Evaluation and Modification of Interventions. 99’1”.“ Almost three-quarters of the Illinois prereferral intervention teams surveyed by Whitten and Dieker (1995) reported that they follow a standardized procedure for analyzing student problems that include these problem-solving steps. Unfortunately, there are data that suggest that these procedures may not be implemented as intended. In one such study, researchers found that the majority of the interventions they studied did not call for teachers to monitor or maintain written records of student performance. Consequently, databased evaluation was not possible (Fuchs, 1987). In another study, results of a review of prereferral intervention plans for the presence of the components of the problem solving processes indicated very few interventions were adequately designed, and even those containing the basic components were of very poor quality (Flugum & Reschly, 1994). 20 Communir 11.: prereferral some me: man) 01.1151 prereferral 6' 4 ' lauti-r e1 3 \lzztter, ar. 5353111611, “it are re‘. ‘13 .. l‘y-“E. “We rem. -r , R s v f ‘ .e't 0- me Communication There is very little research on the actual interactions of professionals during the prereferral process, or with respect to any program or steps within the programs. There is some research that suggests the interactions of team participants are being addressed, and many of the exemplar programs call for training of teams as part of introducing the prereferral intervention processes into schools (Chalfant et al., 1979; Fuchs et al., 1990b; Graden et al., 1985a; Pugach & Johnson, 1988). Over half the teams responding in the Whitten and Dieker (1995) study indicated they had received training on how to function effectively as a team. However, in particular, we do not know whether classroom teachers who are referring students are a part of that team training process. Further, it is not clear from Whitten and Dieker’s description, what team training entailed. The majority of those responding to that questionnaire stated they needed training in the areas of collaborative consultation, strategy instruction, and effective communication. We do not know whether or not these components were part of their initial training, or, if they were part of the training, whether they were sufficient. The literature that emphasizes the need for including the teacher in the problem solving process provides several suggestions related to ensuring communication occurs. These include the consultant asking for the teacher’s observations, inferences, and suggestions (Knoff et al., 1995a; Knoff, Sullivan, & Liu, 1995b); emphasizing that the teacher is responsible for making decisions at each step and providing teachers with options fi'om which to make choices (Graden et al., 1985a; Knoff et al., 1995a; Knoff et al., 1995b); providing strong and positive leadership (Adelman & Taylor, 1998; Graden et al., 1985a); and avoiding direct confrontation or evaluating the teacher’s performance 21 l’lelL'ou. or shat 1... teacher 113. FIG) 151011: {1318 Of a T“. '>“ gunmen miller: EmsBer 0rd: lc. Krista" 117.7331 6.“ (Telzrow, 1997). However, we have no information about whether this actually happens, or what the teacher’s role actually looks like. We do not know what opportunities the teacher has to discuss the interventions, or voice dissent. Nor do we know what provisions are made if the teacher disagrees with the interventions or does not have the time or expertise to implement them. We do not know if teachers find these teams helpfiil, or whether they use the team because it is a prerequisite requirement to initiating a referral to special education. To summarize thus far, relative to our second question, “What are teachers’ perceptions of the problem solving processes of prereferral intervention teams and of the communication that facilitates problem solving?” we know that a specific problem solving process involving four basic components is common to all prereferral programs. However, we know very little about how the processes facilitate communication between team members. We do not know what teachers think about these processes or the communication among team members. Because we do not have this information, we cannot reach conclusions about what teachers perceive is needed to improve team functioning. Intervention Implementation In this section we address literature related to our last two questions, “What are teachers’ perceptions of their own roles on these teams relative to their implementation of interventions?” and “What kinds of supports and resources do teachers state they need in order to make prereferral interventions succeed?” Specifically, we will look at what is known about the interventions themselves. We will review literature on the types of interventions that are most ofien developed, and discuss what is known about their 22 deszgn ll: me till it 30.61011? their use percei‘. e Types 0' fit“; in. - . ..?. ‘t design. We also look at what is known about the implementation of interventions. Then we will describe the literature regarding teachers’ perceptions of their roles in the development and implementation of interventions, and how these perceptions impact on their use of teams. Finally, we will address the types of support and resources teachers perceive they need in order to facilitate intervention implementation. Types of Interventions First, with respect to the types of interventions that are designed, the ones reportedly used most frequently are those that do not require in-depth assessment, planning, or specific knowledge to implement. Examples of the most popular strategies include encouraging and supporting student efforts, and clarifying expectations to the student (Pobst & Patriarca, 1999; Telzrow, 1997). Parent conferences, behavior management techniques, and individual and small group instruction comprises a large portion of the prereferral interventions that are used (Brown, Gable, Hendrickson, & Algozzine, 1991; Carter & Sugai, 1989; Murdick & Fetch-Hogan, 1996). At the same time, research informs us that in order for interventions to be successful, they must be specific, data driven, and designed to meet the needs of the student (Flugum & Reschly, 1994), and appropriate interventions require substantive changes in traditional curriculum and instruction (Ferguson, 1995; Pugach & Warger, 1995). That interventions that emanate from prereferral intervention programs are substantially different from what experts believe constitutes best practice is disconcerting. Yet, we do not know why these teams are not producing interventions that involve the substantive changes in curriculum and instruction that we know are appropriate and more likely to be successful (Ferguson, 1995; Flugum & Reschly, 1994; Pugach & Warger, 1995). 23 lntersc I‘AEK inxfien :haxin: tfuchs Intervention Implementation Second, we know intervention implementation is rarely directly measured (Noell & Witt, 1999). Instead, more typically, teachers are asked whether they have implemented specified interventions, and typically report they have. However, data exist that indicate differences between these reports and direct measures of implementation (Fuchs, 1987; Noell, Witt, Gilbertson, Ranier, & Freeland, 1997). Additionally, research indicates interventions are often not implemented, or, when they are, implementation is inadequate (Flugum & Reschly, 1994; Fuchs, 1987; Noel] et al., 1997). Teachers’ Perceptions of Interventions Third, we know teachers choose not to use particular interventions for a variety of reasons, including the amount of time the programs require, and concerns the interventions will make the problem worse (Johnson & Pugach, 1990). We also know the vast majority of teachers state they have already attempted the interventions recommended by the teams and feel the teams do not explore a sufficient variety of intervention options (Harrington & Gibson, 1986; Inman & Tollefson, 1988). Further, as mentioned earlier, we know teachers may receive support fi'om a teacher consultant or paraprofessional (Bahr, 1994; Carter & Sugai, 1989), but we do not know if they receive the amount or type of support needed to implement the interventions. Finally, we do not know whether teachers raise these concerns during the problem solving process, or, if they are raised, whether attempts are made to resolve their concerns before the team adjoums. At this point, let us summarize what we know relative to our final two questions, “What are teachers’ perceptions of their own roles on these teams relative to the 24 implementation of prereferral interventions in the classroom?” and “What kinds of supports and resources do teachers state they need in order to make prereferral interventions succeed?” Essentially, we know the majority of interventions are poorly designed and inadequately implemented. However, the information we have about why this occurs is very limited, and we have no information about teachers’ perceptions of their roles in these processes. We do not know if high quality interventions are even being developed or recommended. If they are, we do not know whether they are more likely to be implemented. We do not know if teachers would be more likely to implement interventions that are higher in quality - that is, those that reflect best practices - if they had a more active role in intervention development. While we have some idea why teachers choose not to implement specific interventions, we do not know whether, or how, this information is communicated or addressed in prereferral team meetings. We do not know if teachers give up because interventions are too difficult to implement without additional resources or support. In short, we need more information from teachers about their perceptions of the interventions prescribed as a result of the prereferral intervention processes, and what they perceive is needed for them to be able to implement precise, targeted, and successfiil interventions. Justification for the Study This study is important for several reasons: 1) We were unable to find any demographic data on the use of prereferral intervention programs. Further, available information on demographic use of consultation services, in general, is contradictory. Some studies report novice teachers are most likely to use consultative services because they “know they need help” (Mann, 1973). Other 25 studies rep. beause 11.. lS‘.er.ger e meznoati‘l. ‘ C V“. N huff”. C: studies report experienced teachers are more likely to use prereferral intervention teams because they “are secure enough in their positions to be able to accept suggestions” (Stenger et al., 1992). While differences in their results are likely due to sample and methodological differences, our data will not only provide additional information about use of prereferral intervention programs, but will also provide a description of our samples in enough detail to address problems of bias, validity, and replication. 2) We need additional information about teachers who do not use prereferral intervention programs and why they do not use them. We intend to gather preliminary data from teachers who do not or no longer use the teams, determining the extent to which non-use is related to lack of need, use of different resources, or dissatisfaction with prereferral intervention programs. This should help in formulating descriptions of those who use prereferral intervention programs and those who do not use them, as well as casting questions for fixture research about this latter group. 3) We need to know more about teacher involvement in the problem solving and communication processes of prereferral intervention programs. For example, while consultation literature is rife with cautions to be collaborative and to ensure teachers are “included” in intervention development (Graden, 1989; Idol-Maestas, 1983; Sugai & Tindal, 1993; Telzrow, 1997), information regarding whether this is done is limited. This study is designed to address teachers’ perceptions of the problem solving and communication processes, and to provide us with information about the nature of the communication that occurs throughout these interactions. 4) We need to know more about what teachers perceive they need in order to use these teams and implement the interventions that are developed. We have information 26 about Ms, &Gt§m 1.11mi 9113:3311 it their ( \eeli e: these 11' about why teachers choose not to implement specific kinds of interventions (Harrington & Gibson, 1986; Johnson & Pugach, 1990). We also know even though psychologists and teachers agree on the interventions most reasonable to implement in a general education classroom (Pobst & Patriarca, 1999), these interventions are often inadequate in their design and often not implemented (Flugum & Reschly, 1994; Noell & Witt, 1999; Noell et al., 1997). This study is designed to increase our understanding of the nature of these interventions and the contexts in which prereferral interventions are implemented. Ideally this will allow us to adapt and modify procedures and interventions to better meet teacher needs and maximize implementation. 5) Finally, this research uses a combination of quantitative and qualitative research techniques that will allow for greater clarity of our understanding of teachers’ perceptions of prereferral intervention programs. This combination of techniques within the same study is distinctive in the literature on prereferral intervention. Chapter Summary and Limitations of Previous Studies A lot is known about prereferral intervention programs. Much of what we know, is positive. Although specific programs for prereferral procedures are distinctive, they have a lot in common. The goals of the programs are to address the academic, behavioral, social, or emotional needs of the child presented, augment or bone the teacher’s skills for solving future problems, and reduce the numbers of referrals to special education. The programs utilize group problem-solving processes. Most are expert-driven; all are highly structured in the initial steps. Each program involves components of problem analysis and clarification and plan development, and has some provision for evaluation and modification of the plan. However, as these studies illustrate, we know very little about 27 ”'1 US I. r .__1 Fa. teachers’ roles in prereferral intervention programs, other than the fact that, according to reports, they are present. Specifically, we lack knowledge about teachers’ perceptions of their roles on the teams; the problem solving and communication processes utilized by the teams; and their roles relative to development and implementation of interventions. We do not know how their perceptions impact on their use of prereferral intervention programs, or what supports or resources they perceive are needed to improve team fiinctioning and enhance intervention implementation. In short, we do not know enough about teachers and their perceptions of these teams and their roles on these teams to allow us to improve programming in ways that would increase teachers’ use of these programs and ultimately, to improve programming for students. 28 3.- 3.. CHAPTER 3 - METHODOLOGY This study was designed to give us a better understanding of teachers’ perceptions of prereferral intervention teams and the impact these perceptions have on their use of the teams. This chapter presents the research design and methodology used in the collection and analysis of data, beginning with the development of the instruments used and procedures for ensuring informed consent. Subsequent sections include descriptions of the setting and sampling methodology, procedures for collecting data, and descriptions of respondents. Finally, the chapter contains an explanation of the research design and procedures for data analysis. Instrument Development Drawing on the literature on prereferral intervention and related research on problem-solving processes, intervention acceptability, collaboration and consultation, a questionnaire and two interviews were developed, following procedures delineated by Long, Convey and Chwalek (1985). First, a pool of questions was developed around the following topics: 1) teachers’ perceptions of their roles on prereferral intervention teams; 2) the problem solving and communication processes used by these teams and 3) teachers’ perceptions of what supports were needed to ensure implementation of interventions. Questions from the pool were sorted into two categories, 1) questions that could be readily administered in a questionnaire format, and 2) questions that would require responses in detail that were inappropriate for that format. Then, using this pool of questions, a questionnaire and two interviews were developed. The questionnaire was designed for administration to all general education teachers in the sample. The first interview was designed for use with teachers who had 29 use: the te: 13mm: set} the 1: If used the teams during the current and preceding school year, a time period encompassing approximately eighteen months. The second interview was designed for teachers who had used the teams before, but who had not used the teams during the current or preceding school year. Figure 5 depicts the instruments designed and the groups for which each was designed. In this section, we describe the development of these instmments. UESTIONNAIRE All teachers in the sample 1 I NEVER-USERS USERS Teachers who have never Teachers who have used the teams used the teams. at some point in time. I T INTERVIEW w FOR FORMER USERS FOR RECENT USERS Teachers who used teams Teachers who are using the previously but stopped using teams. them. Figure 5 Instruments Develofl and Groups to be Administered Each Instrument Questionnaire Development Items from the pool of questions that were readily administered in a questionnaire format were reorganized and supplemented to produce the questionnaire. The first section of the questionnaire (items 1 through 7) was developed to collect basic demographic data (grade taught, experience, and education) from all teachers who responded. This section was also designed to identify two subgroups: 1) teachers who had never used the teams, and 2) teachers who had used the teams. To simplify matters, we called the two groups 1) 30 "Net er- demos: f , . ‘0.“ .. \b .m 11“. “Never-Users,” and 2) “Users.” We asked Never-Users one question in addition to basic demographic information. Depicted in Table 2, this question was designed to provide us with information about why they did not use the teams. Because our goal was to identify and interview teachers who had used the teams, we then removed “Never-Users” from mestions and Response Choices for Teachers who had Never Used the Teams (Questionnaire) If you have never used the prereferral intervention team in your school. why not? I am a first year teacher I have never needed support services 1 used other support services I understand from other teachers that the learn is not helpful Other The remainder of the questionnaire was designed for “Users,” teachers who had, at some point in time, used the teams. Questions 9 through 11 (see Table 3) were designed to allow us to identify two subgroups of Users, “Former Users,” teachers whose use of the teams had been more than 18 months earlier, and “Recent Users,” teachers who had used the teams within the past 18 months. Although we planned to interview teachers from both subgroups, the decision to differentiate between Former Users and Recent Users was based on two factors. First, we wanted to minimize bias. We chose to interview Recent Users about their experiences with the team because we wanted a sample of teachers who used the teams on an ongoing basis, and thought teachers who had used the teams more recently would be more likely to accurately recall information. We chose to interview Fonner Users because we wanted a sample that was representative of teachers who had ever used the prereferral teams and thought that there might be 31 specific because thatact ~ . l ‘37 Ink". In o. la.\ 993~L.. was.“ Elfin 'rei - ‘. 54.19 ~|{ specific characteristics that would differentiate this group from the Recent Users. Finally, because we could not find any information in the literature about teachers we characterized as “Former Users,” we used this section of the questionnaire to get basic information about why teachers no longer used the team. The remaining questionnaire items (12 through 24 in Table 3) were designed for all Recent Users, but were administered to Former Users as well, for later comparative analysis. Teachers were asked to provide information about the most recent student that they had discussed with the team. Questions in this section also were designed to obtain information about team membership, roles and responsibilities of team members and teachers’ involvement in and agreement with the development and implementation of interventions. Finally, questions were asked about student outcomes and teacher satisfaction. 32 Table 3 Questionaaire Items - Distributed to All Teagers Number Item All Teachers - Ba_s_ic Demtmraphic Data and Use 1) Grade level currently teaching. 2) Number of years teaching that grade. 3) Total years of teaching. 4) Identify your current teaching assignment. 5) Circle the highest degree you have obtained. 6) Are you also certified in special education? 7) Have you ever used the prereferral intervention team in your school? 8) If you have never used the prereferral intervention team in your school. why not? All Users — Data about Recencv of Use 9) During this AND the last school year. how many of your students were referred to this team? 10) If you referred no student during this/last school year. when was the last time you used the team? 11) If you were not involved in the referrals of any students to this team within the last year. why? All Users — Information Most Recent Referral 12) Was the students' problem mainly academic or behavioral? 13) Who attended the meeting? 14) Check the person most responsible for developing interventions. 15) What was your role in the development of the interventions? 16) To what extent did you agree with the plan that was developed? 17) Check the person most responsible for implementing the interventions? 18) If you were primarily responsible for implementing the interventions. did you? 19) If no. why not. OR if you started to. but stopped. why? 20) To what extent was the students' problem resolved? 21) If the students’ problem was not fully resolved. what happened? 22) How satisfied were you with your experience with the team? 23) How likely are you to refer more students to this team? 24) How likely are you to recommend the team to other teachers? Note: See Appendix A for complete questionnaire including response format. Pilot Testing the Questionnaire Prior to administration, the questionnaire was reviewed by two subject-matter experts who evaluated the appropriateness of the items relative to assessing teachers’ perceptions of prereferral programs. These individuals, both members of active prereferral intervention programs, were asked to 1) read each question and describe what they thought the question meant, 2) identify any concerns they had about the clarity of the 33 31’. r.) *1 1's '0 questions, and 3) address whether or not the questions were relevant to teachers’ experiences with prereferral programs. Additionally, the questionnaire was piloted with five teachers, all colleagues of the author. For the questionnaire, teachers were asked to 1) read and respond to the questions; 2) describe what they thought each question meant; and 3) identify any concerns they had about the clarity of instructions or questions. Based on these results, the instrument was progressively revised to resolve any ambiguity in instruction or wording of questions. In addition, another pilot study of the questionnaire was conducted to obtain information about length of time needed for completion and ease of use. This sample, largely a “sample of convenience,” consisted of eleven teachers in an elementary school in a rural school district. Average total time to complete the instrument was four minutes. There were only minor changes for clarity required following expert evaluation and pilot testing of the questionnaire. Criteria for inclusion in the final instrument included item clarity, relevance of the item to the construct being evaluated, relevance of items to respondents in the sample, and item uniqueness relative to other questionnaire items. For a copy of the questionnaire in its entirety, including response format, see Appendix A. Interview Development As mentioned previously, two separate interviews were developed for the groups depicted in Figure 6, one for Former and one for Recent Users, using items fiom the original pool of questions that necessitated detailed responses. In the following sections, we describe the development of the two interviews. 34 USERS l l INTERVIEW FOR INTERVIEW FOR FORMER USERS RECENT USERS Teachers who had not used teams Teachers who had used teams within within previous eighteen months. previous eighteen months. Figure 6 Difl‘eren_t_iation between Team Users and Instruments Develouad Interview Development — Former Users This section describes the development of the interview for Former Users, teachers who had not used the teams within the previous eighteen months. Because we found no information in the literature about teachers who stopped using prereferral intervention programs, we wanted to find out why teachers might stop using the teams. Questionnaire Item 11 “If you were not involved in the referrals of any student to this team within the past eighteen months, why?” was designed to provide this information (see Table 4). These responses formed the foundation for the Former Users Interview. Table 4 Reaponse Choices for Teaalaers who haflrot Used Teams Withjn the Previous 18 Months (Questionnaire) Q-ll If you were not involved in the referrals of any students to this team why? I did not need support services 1 used other support services (which ones? ) I have used the team in the past and did not find it helpful. Other 35 The interview questions designed for the Former Users were open-ended. Essentially, they asked teachers to elaborate on the responses they gave in the questionnaire. Interview questions for Former Users are depicted in Table 5. Table 5 Items Included in tha Interview for “Former Users" Number M You said on your questionnaire that you did not use the prereferral intervention team in your school. giving as your reason: You did not need support services You used other support services (which ones? ) You had used the team in the past and did not find it helpful. Other You did not need support services. 1) Can you tell me why you didn’t need them? 2) Did you find the team helpful when you used it? 3) Will you use the team again? 4) What do you think is needed for those teams to be more effective? 5) What do teachers need to be able to successfully implement the interventions recommended? 6) Is there more I should know about your experience with the team or prereferral process in general? You used other support services thich ones?) 1) Can you tell me why you used (this other service) rather than the prereferral intervention team? 2) Will you use the team again. or continue to use this other resource? 3) Did you find the team helpful when you used it? 4) What do you think is needed for those teams to be more effective? 5) What do teachers need to be able to successfully implement the interventions recommended? 6) Is there more I should know about your experience with the team or prereferral process in general? You used the team in the mst and did not frag it helpful. 1) Could you tell me what happened? 2) Do you think you will use the team again? 3) What do you think is needed for those teams to be more effective? 4) What do teachers need to be able to successfully implement the interventions recommended? 5) Is there more I should know about your experience with the team or prereferral process in general? Other: (Questions to be determined on basis of questionnaire response) Note: See Appendix B for complete interview including response format. 36 i 2ft "'3 j~1 1&6 1's E Pilot Testing the Former Users’ Interview The interview was reviewed by two subj ect-matter experts who evaluated the appropriateness of the items relative to assessing teachers’ perceptions of prereferral programs. These individuals, both members of active prereferral intervention programs, were asked to 1) read each question and describe what they thought the question meant, 2) identify any concerns they had about clarity of questions, and 3) address whether or not the questions were relevant to teachers’ experiences with prereferral programs. We were unable to locate any teachers who met our criteria for “Former Users” with whom we could pilot the interview. Therefore we asked five teachers, all colleagues of the author, to simulate having made a decision to not use the teams. For this pilot study, questions were read aloud by the interviewer. After each question, teachers were asked to 1) describe what they thought the question meant and 2) answer the question. Based on these results, interview questions were revised to resolve ambiguity in directions or wording. Criteria for inclusion in the final instrument included item clarity, relevance of the item to the construct being evaluated, relevance of items to respondents in the sample, and item uniqueness relative to other interview items. For a copy of the Interview for Former Users in its entirety, including response format, see Appendix B. Interview Development — Recent Users This section describes the interview for Recent Users. This interview was designed to have the teacher provide a detailed description of their most recent experience with the team. Interview items are summarized in Table 6. The first part of the interview (Questions 1-10) asked teachers to choose and describe a recent and typical referral to the team. This section asked teachers to describe the students’ age, grade, 37 retention history and gender, as well as whether the student had ever been referred to the teams before. Additional questions elicited reasons teachers had referred the student, and types of interventions they had attempted prior to referring the student. In the second segment of the interview (Questions 11-28), questions were designed to obtain a description of the sequence of activities from initial referral to implementation of team recommendations. Many of the early questions in this section were general (“What happened first?” or “Then what happened?) to avoid asking leading questions. Additional probative questions ensured teachers were asked to provide information related to the steps and procedures used for problem identification, plan development, plan implementation, and procedures for evaluation. Subsequent questions elicited even more detailed information, such as determining what types of information were used in problem clarification, or how the team decided what interventions to use. Finally, teachers were asked to discuss what they thought about the appropriateness and effectiveness of the plan itself, about how the interventions were developed, and what they perceived the outcomes were for the student. In the third and final segment of the Recent Users Interview (Questions 29-33), teachers were asked to comment on their satisfaction with the experience, and the kinds of resources needed to improve team fiinctioning and ensure the implementation of interventions. They were also asked to describe their involvement in overall team fiinctioning in areas such as training, professionalism, and roles. 38 Table 6 Items Included in the In_t_erview for "Recen_t Users“ Number Item Section One - Referral Data 9) 10) How many students do you typically refer to the team in any given year? Was the last student you referred to the team male or female“? Was s/he typical of the students you've referred to the team? Could you give me his/her first name? What grade was s/he in? Had s/he ever been retained? Was this the first time s/he was referred? Why did you refer him/her? (Academic. Behavioral. Both) Could you describe briefly the events or issues that led you to refer him/her? Before you referred him/her. what types of things had you done to try to solve the problem“? Section Two - Procedural Data 11) 12) 13) 14) 15) 16) 17) 18) 19) After you referred the student. but before the meeting. Did anyone collect information? Who? Next. I’d like you think about the meeting itself. I'd like you to tell me about the meeting. Could you tell me what happened first? (Problem identification/analysis/clarification?) Then what happened? (Plan development. procedures for evaluation?) How did the meeting end? (Who was responsible. when. what outcomes would be assessed?) During the meeting. what kind of information did you talk about or look at? What did the team recommend be done? How did the group decide what intervention(s) would be used? What did the planning “look" like? What was your role in deciding what interventions would be used? Was there a written — a formal — plan? If no written plan. did anyone take notes? Who was supposed to implement the plan? If implemented by another. was the plan implemented? Why or why not? If you were the implementer. were you able to implement the plan? Why or why not? Ifyou were able to implement the plan. please tell me what you did In your opinion. to what extent did the plan work? What did you think about the plan? (Did you think it would work? Was appropriate? Had you already tried those interventions? If so. did you tell them? How did they respond?) What did you think about HOW the interventions were developed? Were you adequately involved? Were options available to you if you disagreed with what was recommended? What happened to/with the student? Were his/her problems resolved as a result of this intervention? Did this prevent his/her referral to special education? Section Th_ree - Satisfaction. Otacomes. and Needs 29) 30) 31) 32) 33) Were you satisfied with your experience? What did you think should have happened? Were you treated professionally? What kinds of resources (time. personnel training materials. etc.) do teachers need to be able to successfully implement the interventions recommended? Did you receive training on the functions and use of these teams? What did that consist of? How would you rate your team on a continuum of Very Collaborative to Very Non-Collaborative? Is there anything more about your experience with the team or prereferral process in general? Note: See Appendix C for complete interview including response format. 39 Pilot Testing the Recent Users’ Interview The Recent Users’ Interview was also reviewed by two subject-matter experts who evaluated the appropriateness of the items relative to assessing teachers’ perceptions of prereferral programs. These individuals, both members of active prereferral intervention programs, were asked to 1) read each question and describe what they thought the question meant, 2) identify any concerns they had about the clarity of the questions, and 3) address whether or not the questions were relevant to teachers’ experiences with prereferral programs. Finally, the interviews were piloted with five teachers, all colleagues of the author. For these, questions were read aloud by the interviewer. After each question, teachers were asked to 1) describe what they thought the question meant and 2) answer the question. Based on these results, instruments were progressively revised to resolve any ambiguity in instruction or wording of questions. Criteria for inclusion in the final instrument included item clarity, relevance of the item to the construct being evaluated, relevance of items to respondents in the sample, and item uniqueness relative to other interview items. For a copy of the Interview for Recent Users in its entirety, including response format, see Appendix C. Informed Consent Procedures for obtaining informed consent from all teacher participants were tailored to the specific instrument and distribution method. In all cases, participants were informed their participation was voluntary, and they could refuse to participate or answer questions at any time without penalty. They were also informed their responses would be kept confidential and anonymity would be maintained in any report on the findings. 40 . 1.. 4.1“.“ anfir, V; I 1. ‘l 3.13.1 .1 10.73: I 9 rev“) cl v” ES! '5 Voluntary agreement to participate in the survey was indicated by completion and return of the questionnaire. This information was placed on the first page of the instrument (See Appendix D). Voluntary agreement to participate in the interview was obtained by interviewee signature on the Informed Consent for Interview (see Appendix E). Several steps were taken to ensure confidentiality was protected. First, at the time of the interview, teacher demographic data from the questionnaires was verified and a Consent for Interview form was signed. Each teacher, questionnaire, interview protocol, and audiotape was identified with a code, and teachers’ names were removed from any data. The master list of names and codes are kept locked in a file cabinet in the home of this author and will not be made available to those outside of project staff. Reliability Reliability is the consistency between two measures of the same thing. Our questionnaire consists of single items, and as such, our data are not in format that allows testing for reliability. Relative to our interviews, in qualitative research, particularly in relation to interviews, reliability refers to the degree to which they are reproducible (Payne, 1999). While there is dissension over whether qualitative interviews can, or even should be reproducible, strategies delineated in Payne (1999) were used to ensure appropriate levels of consistency. First, we used semi-structured interviews that involved a predefined range of questions or topics, in order to enable standardization and quantification of results. Second, we included questions that provided confirmation or corroborating responses, and permitted comparisons of interview and questionnaire responses. Third, prior practice by and training of the interviewer (the author) helped ensure consistency in question format and style. Finally, having a second rater evaluate 41 the data, with subsequent discussion that required 100% consensus regarding results, allowed for consistency through scoring agreement (Long, Convey, & Chwalek, 1985). Validity Validity addresses the extent to which a measure assesses what it is intended to measure. The validity of our measures was evaluated relative to content-validity. Evidence of content-related validity is obtained when it can be determined the instruments provide sufficient coverage of the area of interest as well as through evaluation of the appropriateness of the format of the measures (Fraenkel & Wallen, 1996). To evaluate content validity, we explored how well our questions sampled our area of interest - teachers’ perceptions of prereferral interventions teams. In questionnaire and interview development, our task involved defining the content domain as specifically as possible, then devising a sampling process of this set to choose questionnaire items or interview questions. Content-related validity of the questionnaire and interview were addressed through examination of the instruments by persons familiar with pre-referral intervention programs and the goals of this study, using procedures delineated in Fraenkel and Wallen (1996). These included providing two examiners with a definition of what was intended for the study and a description of the sample, as well as copies of the instruments themselves. The examiners rated the objectives of the study against the questions on the questionnaire and interviews, as well as assessed the adequacy of the format. Items were modified and returned to the evaluators until a consensus was achieved. Fraenkel and Wallen (1996) describe several threats to internal and external validity that can introduce bias and limit the generalizability of the results that were felt 42 to be particularly pertinent within the scope of this study. The first source of potential bias involves subject characteristics such as satisfaction, attitude, and experience with prereferral intervention teams, when those chosen for the sample are different than those who are not. A second source of potential bias is called location threats. These involve the potential impact of the location where the instrument is administered, particularly if the respondent is concerned about his or her responses being seen or overheard. A final source of error is data collector bias, characteristics or dispositions of the data collector, such as encouragement or disparagement, which may make certain responses more likely. The following techniques were used to compensate for these potential sources of error. Bias related to subject characteristics were evaluated by comparing teachers who agreed to be interviewed and those who did not, on a variety of variables including education, experience, satisfaction and outcome data. Location threats were minimized by ensuring interviews were conducted in quiet, private settings within the teachers’ school, and all data were carefully collected and confidentiality was maintained in the handling of data where teachers could be identified (such as audiotapes). Standardizing all procedures and ensuring the data collectors were adequately trained and experienced in procedures such as interviewing aided in reducing data collector bias. Furthermore, the use of probes during the interview allowed us to obtain clarification of responses and request additional detail when responses were unclear. Finally, triangulation was used to help establish validity in the qualitative research. Defined by Payne (1999) as obtaining evidence from more than one source, the questionnaire and interview serve as sources of data to be used for triangulation. 43 Characteristics of the Sample In this section, we describe district demographics and the reasons for its selection as a study site, including information about the characteristics of the prereferral intervention programs in the district. We also describe the sampling methodology used. Setting This study was conducted in 18 of 34 elementary schools (grades K-5) in a midsize urban school district. Demographically, the district has a population of approximately 19,000 students, and 950 teachers. Over half the students in the district are from racially and ethnically diverse and/or economically at-risk environments, 30% are African-American, 17% are Hispanic, 6% are Asian, and 1% are Native American. Over half of the students (54%) qualify for free lunch. The district was selected for several reasons. First, prereferral intervention teams are mandated at the elementary level and have been in place for several years (eight to ten, depending on the building). Second, team representatives reported that teams received training in consultation and utilize a model based on multidisciplinary problem solving. The stated purposes of the teams are consistent with those of prereferral intervention programs described in the literature. That is, team goals are to provide immediate and ongoing support to teachers in meeting student needs, enhance teachers’ skills for working with difficult-to-teach students, and reduce the numbers of referrals to special education. Finally, the district was chosen because of its status as an urban school district, an environment in which effective intervention and professional development programs are particularly critical (Burstein & Sears, 1998). 44 Sampling Methodology Sample size was determined by following recommendations in Henry (1990) and Borg, Gall and Gall (1993). First, we determined a return of 45 questionnaires would be necessary to obtain statistically significant correlations at the .01 level with obtained correlations of .15 or higher (Borg, Gall, & Gall, 1993). However, before collecting data, we used the following information to arrive at a target sample pool size: First, based on this author’s experience with prereferral intervention programs in another district, we anticipated approximately half of the teachers in a building would use the team in a given year. Second, in a previous survey we conducted of teachers in four schools in the target district we obtained return rates of 75% (Pobst & Patriarca, 1999). We concluded that following comparable methodology in data collection, we should be able to generate return rates of at least 60%. Third, we also needed to ensure a large enough interview pool — composed of those who returned the questionnaires who indicated willingness to participate in the second part of the study. Based on the results of a study that utilized similar methodology (Johnson & Pugach, 1990), we anticipated approximately 30 percent of those who responded to the questionnaire would be willing to participate in an interview. Finally, adjustments needed to be made to forestall problems with ineligibles and nonresponders. Taking all these factors into consideration, we concluded that an optimal, yet efficient sample size would be 200. If our calculations were correct, it would provide us with the 100-120 questionnaires and 20-30 interviews we needed. School sites selected for this study were identified by the School District’s Office of Research and Planning. This was done by identifying every other building on an alphabetic list of 32 of the 34 elementary schools in the district (teachers in two of the 45 schools in the district were involved in major research projects, and were, as such, excluded from other research activities). Then, the psychologist from each building was identified, as representative of the team assigned to that building. One school was added to the list to ensure all the psychologists in the district were represented in the sample (each of seven psychologists was assigned to at least two schools). This resulted in the selection of 17 schools. The principal of each school was contacted to describe the study and request permission to approach the teachers in their buildings. Principals of four of the 17 selected elementary schools denied entry into their buildings. Two of those principals cited lack of time and interest on the part of faculty. The other two stated they did not believe the teachers in their schools would provide candid responses about their perceptions of the teams through the proposed methodology, and requested they be excluded from the study. Thus, the original sample size consisted of thirteen elementary schools. Data Collection Procedures This study was composed of two distinct methodologies, questionnaire and interview. The methodology described in this section for conducting survey research follows recommendations of Borg, Gall, and Gall (1993), Mangione (1995), and Lockhart (1984). The methodology described in this section for conducting interview research follows recommendations of Bogdan and Biklen (1998) and Payne (1999). In this section we describe data collection procedures used for each method and the response rates for each. 46 Distribution and Response Rates for Questionnaires Questionnaires were distributed in 13 buildings. The procedures used depended on the building. Three different sets of procedures for collecting questionnaire data were used in this study: 1) presentations at staff meetings, 2) individual contacts of teachers by building representatives, and 3) distribution by principals in staff mailboxes. In this section we discuss each of these methods of distribution and the response rates for each method. Table 7 summarizes the distribution and return rates for each type of distribution. Presentations at staff meetings: In six of the 13 schools, the study was described to teachers at regularly scheduled staff meetings. At that time, written descriptions of the study and copies of the questionnaire were distributed for completion and collection. Principals identified teachers who were not in attendance at the staff meetings. Information about the study and a copy of the questionnaire was coded and distributed to teachers identified in this manner via school mailboxes. An addressed, stamped envelope in which to return the questionnaire was provided. Finally, follow-up letters with a new copy of the questionnaire and a hand-written request they complete and return the questionnaire were mailed to teachers who had not returned it within two weeks. In the six buildings where we distributed and collected questionnaires following these procedures, questionnaires were distributed to 57 teachers. According to the principals in their buildings, a total of seven teachers were not present at staff meetings. Materials were distributed to these teachers with addressed, stamped envelopes in which to return the questionnaires, along with personalized notes requesting their participation. Follow-up letters were sent to these teachers who had not returned the questionnaires 47 within three weeks. F ifty-six usable questionnaires were returned, representing 88% of the teachers in those schools. Building Representatives: In four of the 13 buildings, principals were not able to accommodate the project during staff meetings. In these buildings, questionnaires were distributed and collected by teacher candidates interning in the buildings. These individuals distributed the questionnaires to teachers, either personally or through school mailboxes, and collected completed questionnaires. They also conducted follow-ups and contacted non-responders personally. In these four buildings where a building-based representative distributed and collected materials, questionnaires were distributed to 55 teachers. Forty-five usable questionnaires were returned, representing 82% of the teachers in those schools. Building Principal: In the remaining three buildings, principals agreed to distribute questionnaires with self-addressed stamped envelopes via school mail, but would not provide staff lists to allow us to determine how many teachers in the school met the selection criteria (firll time general education). In these buildings, principals asked for and received a specific number of questionnaires; each with an accompanying stamped envelope addressed to this author, and distributed them via staff mailboxes. There was no provision for follow-up in these three schools. In these three buildings, questionnaires were distributed to approximately 31 teachers. Seven usable questionnaires were returned, re resentin rou hl 23% of the teachers in those schools. P 8 8 Y 48 Table 7 Questionnaire Distribution and Return Rates per Type of Disaribution Method Typ_e of Number of Initial Initial Follow Usable Percent Distribution Buildings Distribution Return Lip Return Total Return Staff Meeting 6 57 56 7 4 56 88% Building Representative 4 55 46 NA NA 45 82% Mailboxes 3 3 l 7 NA NA 7 23% TOTAL 13 143 109 7 4 108 72% In total, 150 questionnaires were distributed to teachers in 13 buildings. One hundred and eight usable questionnaires were collected, representing an overall return rate of 72%. Follow Up Study of Nonresponders Although there is no minimum agreed-upon standard for an acceptable response rate, Babbie (1989) noted a very good response rate is 70%, a good response rate is 60%, and an adequate response rate is 50%. Accordingly, the 72% response rate to the questionnaire can be considered very good. However, Lockhart (1984) indicates response rates above 80% are necessary to ensure there are no significant differences between responders and nonresponders that could yield biased results. Because our procedures resulted in response rates below the 80% level, we spoke with several nonresponders. These teachers were identified during staff meetings, when questionnaires were being distributed by building representatives, and during interviews, when teachers who wanted to comment on the teams approached this author. The goal of this follow-up was to ensure there were no significant differences between responders and nonresponders that could yield biased results. According to Lockhart (1984) contacting ten percent of 49 nonresponders constitutes an appropriate sample. Of the 42 nonresponders, nine teachers (21%) provided this information. They were asked: a) whether or not they had ever used the teams, b) when they had last used the teams, and c) whether they had been satisfied or dissatisfied with the teams. No additional information about the non-responders was obtained. Finally, it should be noted that in 10 of the 13 schools in which we collected data, response rates were above the 80% response rate recommended by Lockhart (1984). Overall rates were reduced by lower response rates in those schools where entry was limited. We do not know in what ways teachers might differ in buildings where principals allow different levels of access. Data Collection - Interviews Teachers who were willing to be interviewed indicated this on the last page of the questionnaire (See Appendix F). These teachers formed the pool of interview candidates. We had intended to interview representative samples of Former Users and Recent Users. However, only one teacher fell into the Former Users category, so we never utilized the Former Users Interview we designed for this study. All teachers who indicated they were willing to consider being interviewed were contacted, to: a) answer any questions they had about the interviews, b) affirm their willingness to participate, and c) arrange a time and place for the interview. Interviews were audiotaped and transcribed for analysis. Response Rates - Interviews Of the 108 teachers who indicated on their questionnaires that they had used the prereferral intervention teams in their buildings, 38 (35%) indicated that they were willing to participate in the interviews. We attempted to contact all 38 of these teachers, 50 but were unable to reach five of them despite repeated attempts to do so. Of the remaining 33 teachers, three scheduled and then cancelled interviews, citing work, school, and family conflicts. Ultimately, 30 teachers from the initial group of 38 were interviewed. These teachers taught in twelve of the thirteen school buildings in our sample. Follow Up Study - Questionnaires High return rates from the thirteen buildings participating in the study meant we did not need to select additional buildings to replace the schools where principals opted not to participate in the study to obtain an adequate response rate. However, preliminary data analyses revealed that obtaining at least 30 more questionnaires would permit more powerful statistical analyses. Although we considered postponing this phase, leaving it for future research, teacher interest and willingness to participate was such that we decided to collect the extra questionnaires. We calculated that to ensure sufficient responses we needed to distribute questionnaires to teachers in five additional buildings. These buildings were selected by calling the remaining schools on the alphabetic list. When we were able to reach the school principal by telephone, we described the study and asked if they would be willing to participate. If we were not able to reach the principal, we called the next building on the list. We continued calling schools on the list and talking with principals until we had permission to enter the five additional buildings we needed. In these five buildings, questionnaires were distributed and collected by student teachers or university supervisors who had worked in and/or retired from the buildings. These individuals distributed the questionnaires to teachers, either personally or through school mailboxes, and collected completed questionnaires. They conducted 51 follow-up requests and accounted for non-responders personally. Teachers in these schools were asked to fill out the questionnaires. However, because our goal was simply to increase the number of questionnaire responses, these teachers were not asked to participate in interviews. Using these procedures, 64 questionnaires were distributed to general education teachers in the five selected schools. We received usable questionnaires from 46 teachers (72%). As described earlier, this is a very good response rate (Babbie, 1989) Adding questionnaires from these five schools to those from the original thirteen schools, then, a total of 214 questionnaires were distributed to general education teachers in 18 buildings. Total usable responses were received from 154 teachers (72%). Distribution and return rates from these distributions are depicted in Table 8. Table 8 Questionnaiae Distribution and Return Rates —- Initial and Follow Up Distributions Distribution Number of Buildings estionnaires Ugble Return Percent Retum Initial Distribution 13 150 108 72% Follow Up Distribution 5 64 46 72% TOTAL 18 214 154 72% Finally, we wanted to consider how representative our sample was of the teachers in the district, a factor in assessing potential response bias. As the Office of Research in the district does not provide access to demographic data about teachers, we used the most recent district directory to obtain general estimates of the number of teachers at each grade level within the district. Unfortunately, the directory was last published in 1998 and we know there were numerous personnel changes from the 1998-1999 to 1999-2000 52 school year. Because of this, we question the accuracy of these distributions and no statistical analyses were conducted on these data. However, using the frequencies of teachers at each grade level from the directory, we made cautious comparisons to the distributions of teachers in our sample. Delineated in Table 9, it appears that our sample is generally representative of the district population in terms of grade level taught. Table 9 Distributioa of Teachers in the District According to Grade Taught (Questionnaire and Directory‘ Data) Elementary Teachers in the Elementarv Teachers in the Sample District n=454 * (n=154) K 71 15.6 25 16.2 1 88 19.4 25 16.2 2 78 17.2 29 18.8 3 64 14.1 22 14.3 4 51 11.2 26 16.9 5 51 11.2 16 10.4 Multi-grade 5 I l 1.2 10 6.5 Missing -- - l .6 *Dism'ct data are from the year preceding the study (1998-1999). Research Design This is a descriptive study, using two complementary methodologies, questionnaire and interview. The study has two stages. In the first stage, teachers were administered a questionnaire that assessed their use of the prereferral intervention teams in their school and provided information related to their roles and responsibilities on the team, as well as to the outcomes of the interventions designed. Questionnaire results were analyzed to identify, describe, and examine differences between the following subgroups of teachers: 1) Never-Users and Users, 2) Former Users and Recent Users, and 3) those Interviewed and Not Interviewed. Never-Users were eliminated from the study after analyzing the reasons they did not use the team. Then, questionnaire results were 53 examined to address the relationship between team use and teacher perceptions of their participation on the teams, their role in the development and implementation of interventions, and their expressed satisfaction with the experience and outcomes. In the second stage of the study, a subset of the teachers who referred students to the teams were interviewed to obtain information about their experiences with prereferral intervention programs. Interview results were analyzed to provide additional information about teachers’ perceptions of their roles in the prereferral intervention processes, and about what they perceive is necessary to improve team functioning. Information obtained from each method was used to explain, clarify, and supplement information obtained fi'om the other. Data Analysis The questionnaire and interview generated different types of data that called for a variety of statistical tests. In most cases, data were condensed into categories, to accommodate analyses. In general, data were addressed in three ways. First. several of the sections involved nominal data that allowed the use of frequencies and percentages. Second, some data were analyzed using stepwise logistic regression techniques outlined in Hosmer and Lemethow (1989). Logistic regression allows us to predict the presence or absence of a characteristic or outcome based on values of a set of predictor variables, where the independent variable is dichotomous. Because alpha levels of .05 are generally considered too stringent for use in these types of chi-square analyses, we used the recommended levels of p < .20 (Hosmer & Lemethow, 1989). Additional data were analyzed using qualitative methods outlined in Bogdan and Biklen (1998). Qualitative techniques allow us to obtain a depth and breadth of understanding of the data that is not 54 possible using quantitative techniques in isolation. They allow us to investigate the quality of relationships, activities, and situations, and to obtain explanations and generate additional questions and hypotheses about the data. First, where responses could be quantified (i.e. “Did you receive training in prereferral intervention?”) data are reported in frequencies and percentages. Second, inter-rater reliability was addressed by having two raters independently evaluating transcripts, using these to identify recurring themes, patterns, and critical issues within each of the three areas of scrutiny — roles, processes, and outcomes. The raters then discussed their findings and reached consensus in developing a master list of categories. Interview transcripts were examined again, grouping and regrouping data, and sorting information into subsets by category. Emergent theories were evaluated against the data and data scrutinized to identify alternate explanations. Research Questions This study was designed to answer several questions. The sections below identify the four major research questions. Each question is followed by the items from the questionnaire and interviews designed to collect the data to answer each question, as well as the statistical analyses used. Complete copies of the study instruments can be found in the Appendices A and C. QUESTION #1: Who uses prereferral intervention teams? Analysis: This question was operationalized using the responses from items from the first two sections of the questionnaire. Table 10 has the specific items used for analyses. Data are reported in frequencies and percentages of the responses for each choice for the item. Chi-square tests are used to examine differences between subgroups. 55 Table 10 mestionnaire Items AddressingTeacher Use of Terms Number ltern Q-l Grade level currently teaching. 0-3 Total years of teaching. Q-5 Circle the highest degree you have obtained. Q-6 Are you also certified in special education? Q-7 Have you ever used the prereferral intervention team in your school? (Yes/No) 0-8 If you have never used the prereferral intervention team in your school. why not? Q-9 During this school year (1999-2000) AND last (1998-99) school year. how many of your students were referred to this team? Q-10 If you referred no students during this time. when was the last time you used the team? Q-ll If you were not involved in referrals of any students to this team within the last year. why? Note: Item number refers to Questionnaire (Q) or Interview (1) and the corresponding item on that instrument - See Appendices A and C for complete copies of the instruments. QUESTION #2: What are teachers’ perceptions of the problem solving processes of prereferral intervention teams and of the communication that facilitates problem solving? Analysis: The first portion of this question was operationalized using questionnaire and interview instrument items designed to assess problem solving processes used by the teams. Specifically, these related to problem identification, plan development, and procedures for plan implementation and evaluation. The second portion of this question related to teachers’ perceptions of the communication that facilitates problem solving. This was operationalized using the questionnaire and interview instrument items designed to assess communication and teacher opportunities to provide input, make decisions, discuss options and provide dissenting opinions. These items are depicted in Table 11. 56 Table II mestionnaire and Interview Items Addressing Problem Solving and Communication Number Item 1 After you referred student. but before meeting. did anyone collect information? If yes. who? 6 During the meeting. what kind of information did you talk about or look at? 3 Could you tell me what happened first? (Problem identification/ analysis/ clarification?) 4 Then what happened? (Plan development. procedures for evaluation?) 5 How did the meeting end? (Who was responsible. when. what outcomes would be assessed?) 8 How did group decide what intervention(s) would be used? What did planning “look“ like? 1-19 What was your role in deciding what interventions would be used? Q-14 Who was the person most responsible for developing interventions? Q-15 What was your role in the development of the interventions? Q-16 To what extent did you agree with the plan that was developed? I-26 What did you think about the plan? (Did you think it would work? Was it appropriate? Had you already tried these interventions? If so. did you tell them? How did they respond?) [-27 What did you think about HOW the interventions were developed? Were you adequately involved? Were options available to you if you disagreed with what was recommended? I-32 Rate team on a continuum of Very Collaborative to Very Non-Collaborative/Expert Based? Note: Item number refers to Questionnaire (Q) or Interview (1) and the corresponding item on that instrument - See Appendices A and C for complete copies of the instruments. Questionnaire data are reported in frequencies and percentages of the responses for each choice for each item. Interview data are addressed in two ways. First, data are reported using frequencies and percentages of the responses. Then, the raters’ conclusions are discussed, using comments and quotes from teachers to illustrate, interpret, and explain both questionnaire and interview data. Finally, we took the data from Question 1 and examined whether, and to what extent, these demographic data were predictive of whether teachers indicated they perceived themselves as involved in the development of interventions. Whether the teachers’ role was one of involvement was operationalized using responses to the last two choices on Questionnaire Item 15 that indicated teachers had a collaborative or leadership role (See Appendix A). Data from the questionnaire used to obtain demographic data 57 were consolidated for analyses, according to criteria in Table 12. Data were analyzed using stepwise logistic regression techniques outlined in Hosmer and Lemethow (1989). Table 12 Questionnaire Variables Used to Address Correlates of Involvemeau Item Indepandent Variable mpendent Variables Q-15 Was the teachers' role 1) Demographic data: collaborative or did Years of experience (0-5. 6—20. 21+) they assume a Education (BA-BA+. MA-MA+) leadersz role? Grade taught (K-I. 2-3. 4-5) 2) Refenal data Number of students (I.2.3.3+) Type of Problem (Academic. Behavioral. Both) QUESTION #3: What are teachers’ perceptions of their own roles on these teams relative to the implementation of prereferral interventions? Analysis: This was operationalized using questionnaire and interview instruments items depicted in Table 13 designed to address intervention implementation. Questionnaire data are reported in frequencies and percentages of the responses for each choice for each item. Interview data are addressed in two ways. First, data are reported using frequencies and percentages of the responses. Then, the raters’ conclusions are discussed, using comments and quotes from teachers to illustrate, interpret, and explain both questionnaire and interview data. 58 Table 13 Questionnaire and Interview Items Addressing Interveruion Implementation Number Item 1-7 0-17 1-21 Q-18 1-22 1-23 1-24 Q-19 1-25 I-26 What did the team recommend be done? Who was most responsible for implementing the interventions? Who was supposed to implement the plan? If you were primarily responsible for implementing the interventions. did you? If implemented by someone else. was the plan implemented? Why or why not? If you were the implementer. were you able to implement the plan? Why or why not? If you were able to implement the plan, please tell me what you did If no. why not. OR if you started to. but stopped. why? In your opinion. to what extent did the plan work? What did you think about the plan? Note: Item number refers to Questionnaire (Q) or Interview (1) and the corresponding item on that instrument - See Appendices A and C for complete copies of the instruments. QUESTION #4: What kinds of supports and resources do teachers say they need in order to make prereferral interventions succeed? Analysis: This was operationalized by using the interview instrument developed for the study to gather statements from teachers about what was needed in order to make interventions succeed. These questions are depicted in Table 14. Data are analyzed qualitatively, using frequencies and percentages of responses, and illustrated and interpreted using the two raters’ conclusions, and comments and quotes from teachers. Table 14 mstionnaire and Inaerview Items Addressing Supmrts and Resources Needed Number Ite [-30 1-31 [-33 What kinds of resources (time. personnel training. materials. etc.) do teachers need to be able to successfully implement the interventions recommended? Did you receive any training on the functions and use of prereferral intervention teams? What did that consist of? Is there anything else you think 1 should know about your experience with the team or prereferral process in general? Note: Item number refers to Questionnaire (Q) or Interview (1) and the corresponding item on that instrument — See Appendices A and C for complete copies of the instruments. 59 Additional Analyses — Satisfaction Finally, we had only intended to look at satisfaction data comparatively — in our planned analyses of differences between Former and Recent Users. The reader will recall that 127 out of 128 Users in our sample were Recent Users, and as such, no comparative analyses could be made. Nevertheless we felt that examination of these data might be useful. Therefore, post hoc, we added a fifth set of analyses related to satisfaction, based on teacher responses to the questionnaire and interview items listed in Table 15. Table 15 Questionnaire and In_terview Items Addressing Satisfaction No. Item Q-22 How satisfied were you with your experience with the team? 1-29 Were you satisfied with your experience? Note: Item number refers to Questionnaire (Q) or Interview (1) and the corresponding item on that instrument - See Appendices A and C for complete copies of the instruments. Questionnaire data are reported in frequencies and percentages. We then took the data in Question 1 and examined whether, and to what extent, these demographic data were predictive of whether teachers said they were satisfied with their experiences with the team. Analyses were completed using stepwise logistic regression procedures to determine the relationship whether specific demographic variables predict satisfaction. They are supplemented with interview data, reported in frequencies and percentages. Limitations There are a number of factors that may limit the generalizability of the results of this study. First are the sample-specific limitations. The extent to which the respondents in these samples are representative of other groups of teachers in urban districts (population generalizability) will be subject to replication (Borg et al., 1993). Further, the 60 extent to which prereferral intervention programs in this district are representative of prereferral intervention programs in general (ecological generalization) is also subject to replication. Another limitation involves interpretation bias when respondents use their own interpretations of questionnaire items in the absence of clear and specific item definitions (Borg et al., 1993), or as subjects give information they believe the examiner wants to hear (Yin, 1994). Further, the data are self-report in nature, and respondents relied on memory to respond to questions. We do not know whether what respondents report are what they actually experienced. Observation of prereferral team interactions and teachers implementing interventions would be needed to ascertain whether or not respondents were answering in a manner consistent with their actual performance (Borg et al., 1993). Finally, when and how a teacher chooses to use an intervention is a complicated process. Variables such as the nature and specificity of the presenting problem, level of severity, comorbidity, or external pressures affect strategy use. How responses to both questionnaires and interviews might differ if the teachers described different students, or interactions with individual team members, remains to be addressed in future research. Chapter Summary This chapter presents the research design and methodology used in the collection and analyses of data. In the first section, development of a questionnaire and two interview instruments and procedures for obtaining informed consent were explained. In the second section, we presented descriptions of the setting and sampling methodology. Then, we explained procedures for collecting data and provided descriptions of respondents. Finally, we presented the research design and procedures for data analysis. 61 The next chapter presents results based on the analyses explained in this chapter. The final chapter discusses the implications of those results for improving prereferral intervention programs. 62 CHAPTER FOUR — RESULTS The primary purpose of this research is to analyze teachers’ use and perceptions of prereferral intervention programs. In this chapter we present the results of the quantitative and qualitative analyses of questionnaire and interview data pertaining to our basic research questions regarding teachers and their roles, procedures and communication patterns that facilitate successful problem solving and intervention development, implementation, and outcomes. Question One: Team Use In this section, we provide data related to our first question, “Who uses prereferral intervention teams?” First, we present demographic data about teachers in our total sample. Then, we discuss our analyses of subgroups of teachers: 1) Never—Users and Users, 2) Former Users and Recent Users, and 3) Users who were Interviewed and Users who were Not Interviewed. Finally, we present and discuss the reasons given by Never- Users for not having used the teams. Characteristics of the Sample In this section, we present characteristics of the total sample, describing grade taught, years of experience, and educational background of the 154 general education teachers who completed and returned the questionnaires. The distribution of teachers at each grade level was roughly equal for grades kindergarten through four (between 14% and 19% of the teachers in the sample were at each grade). There were relatively fewer fifth grade teachers in the sample (10.4%). However, as described in Chapter Three, this is generally commensurate with the grade level distributions within the district. For some reason all the schools in this district have fewer 5th grade classes than kindergarten 63 through fourth grades. The reasons for this are not known. There were also fewer teachers in multigrade classes in our sample (6.5%) than in the district (11.2%). We do not know why this occurred. It may be because the distribution of these types of classes across the district was uneven and consequently there were fewer multigrade classes in the buildings we entered. Or, it may be that teachers in the multigrade classes were less likely to complete and return the questionnaires. Teachers in the sample reported years of experience ranging from one to thirty- nine, with a mean number of years of experience of 14.41. Teachers were evenly distributed between those who had BA degrees or BA degrees with some coursework beyond the degree (48.7%) and those who had an MA degree or higher (48.1%). Nine (5.8%) of the teachers in the group had an endorsement in special education in addition to their general education credentials. Relative to team use, 26 teachers (17%) said they had never used the teams, 128 (83%) said they had used the teams. A summary of these data is presented in Table 16. 64 Table 16 Characteristics of the Sample (n=154) (Queationflire Data) Fraguencv Earpaut G_rade Teacher is Currently Teaching K 25 16.2 1 25 16.2 2 29 18.8 3 22 14.3 4 26 16.9 5 _ 16 10.4 Multigrade 10 6.5 Missing 1 0.6 Total Years Teaching 0-5 34 22.1 6-10 38 24.7 11-20 29 18.8 21-30 41 26.6 31+ 9 5.8 Missing 3 1.9 Level of Education BA 11 7.1 BA+ 64 41.6 MA 58 37.7 MA+ 16 10.3 Missing 5 3.2 Special Education Certified? No 142 92.2 Yes 9 5.8 Missing 3 1.9 five vou ever used the team? No 26 16.9 Yes 128 83.1 Subgroup Analyses As discussed in Chapter Three, we divided our responders into subgroups. We partitioned them in three different ways: 1) Never-Users and Users, 2) Former Users and 65 Recent Users, and 3) Users who were Interviewed and Users who were Not Interviewed. In this section, we describe these subgroups. Subgroup l - Never Users and Users: First, we divided responders into two subgroups, Never-Users and Users (see Figure 7). We did this because we wanted to examine teachers’ experiences with the teams, and planned to remove Never-Users from the second phase of the study. Of the 154 responders, 17% were Never-Users (n = 26), and 83% were Users (n = 126). RESPONDENTS (n = 154) 1 l I NEVER USERS USERS (n = 26) (n = 128) Figur_'e 7 Relationship to Reapondents of Subgroup 1: Never Users and Users We compared Never Users and Users on basic demographic data: 1) grade taught, 2) years of experience, and 3) level of education. Subgroup 2 - Former Users and Recent Users: Second, we divided the Users into two subgroups, Former Users and Recent Users (see Figure 8). We did this because we wanted to ensure we interviewed a representative sample. We also wanted to account for potential bias, as we were concerned Former Users might not recall the information asked for in the questionnaire. Of the 128 Users, however, only one (0.8%) was a Former User. 66 RESPONDENTS (n = 154) l USERS (n = 128) l l l FORMER USERS RECENT USERS (n =1) (n =127) _F_igu_re 8 Relationalup to Regpondents and Users of Subgroup 2: Former Users and RecenLUsers The remaining 127 teachers (99.2%) had all used the team within the previous eighteen months or were currently involved in referrals. Because we did not have enough Former Users, we dropped our original plan to conduct a comparative analysis of the data by recency of use (Former and Recent). Subgroup 3 - Users Interviewed and Users Not Interviewed: We again divided the Users into two subgroups, this time, those who were Interviewed and Not Interviewed (see Figure 9). We examined this relationship because we wanted to determine whether teachers who were interviewed were representative of all teachers who used the teams. 67 RESPONDENTS (n = 154) l USERS (n = 128) l l l INTERVIEWED NOT INTERVIEWED (n = 30) (n = 98) Figure 9 Relationship to Reapondents arui Users of Subgroup 3: Users Ipterviewed fld Not In_tarviewed We looked at whether these two subgroups — those interviewed and those not interviewed — were similar or different using several variables, listed in Table 17. Table 17 mestionnaire Items Used for Analysis of Subgroup Differences: Users Mentiewed aria Not Inaerviewed Q-l Grade Taught Q-3 Years of Experience Q-5 Level of Education Q-9 Number of Students Referred Q-14 Teacher most responsible for developing interventions? Q-15 Did teacher consider role to be collaborative/leadership? Q-17 Was teacher primarily responsible for implementing interventions? Q-20 Was students’ problem fully or partially resolved? Q42 Was the teacher very or somewhat satisfied? Results of the Subgroup Analyses Subgroup I - Never-Users and Users: While we planned to exclude Never- Users from both phases of the study, we did obtain some data about these teachers and compared the two subgroups relative to grade taught, years of experience, and educational level. Before moving on, we want to present the results of these comparisons. First, 26 of the 154 teachers we surveyed ( 16.9%) never used the teams. We found Never 68 Users and Users similar in grade taught and educational level. They were statistically significantly different only for years of experience, x2 (35) = 71.922, p = .000. Comparisons were made using Chi-square Tests. The mean years of experience for the 26 teachers who had never used the teams was 8.58; whereas the mean years of teaching for the 128 teachers who had used the teams was 15.57. We wondered why we found this difference, and looked at the additional data we had about Never Users. We had asked Never Users to indicate why they had never used the teams. Their responses are listed in Table 18, below. Table 18 Reasons Given by Teachers for Never Having Used the Teams (Questiongire Dafl) 13% F requencv (n=26) Percent I am a first year teacher 10 38.5 I have never needed support services 8 30.8 I used other support services 3 11.5 I understand from others the team is not helpful 1 03.8 Other 4 15.3 In particular, we noticed that ten of the 26 Never-Users (38%) indicated they were first year teachers. In fact, no first year teacher who completed and returned the questionnaire reported ever having used the team. In order to determine the significance of this finding we eliminated first year teachers’ data from the Never-Users data. We then compared the remaining 16 Never-Users to Users on the demographic variables (grade taught, years’ experience, and educational level), again using Chi-square tests. We found no statistically significant differences between the two groups for grade taught (x2 (10) = 2.813, p = .985); for years of experience (12 (33) = 40.983, p = .160); or for educational 69 level (12 (3) = 2.083, p = .555). Consequently, when the first year teachers were not included in the Never Users subgroup, the Never Users and the Users looked the same. Once the first year teachers were removed from the Never Users subgroup, 16 teachers remained. Of these remaining 16 Never Users, eight (50%) identified they had never needed support services. The reasons for this are unknown. Three of the remaining 16 Never Users (18.8%) used other support services, specifically, a Student Assistance Provider, a School Counselor, and a Special Education Teacher. Four (25%) did not specify their reasons for not having used the team, and one (6.3%) indicated they understood from others the team was not helpful. These results are presented in Table 19. Table 19 Reasons Given for Never Having Used Teams - First Year Teachers Excluded (Questionnaire Data) m Fregueng' Percent Percent of the (n=l6) (n=l6) Sample (n=26) I have never needed support services 8 50.0 30.8 I used other support services 3 18.8 1 1.5 I understand from others the team is not helpful 1 6.3 03.8 Other 4 25.0 15.3 These results indicate the only variable that differentiated Never Users from Users is ““first year” status. We are not sure why first year teachers never used the team. However, there are several logical possibilities. First, it is possible first-year teachers had not yet learned that these teams existed in the building. It is also possible they were so overwhelmed with new job responsibilities they did not have the resources to complete a referral. Or, they may still have been learning to differentiate between what is typical and atypical behavior for a given grade level, and did not feel confident enough to submit a referral. Everything we know about first year teachers suggests they have more difficulty 70 handling students and need more help and support than do more experienced teachers. Because of this, the fact they are not using the teams is puzzling. Subgroup 2 - Former Users and Recent Users: Because there was only one Former User in the sample, we did not conduct any subgroup analyses on F onner and Recent Users. We will proceed with reporting our analyses of the third subgroup. Subgroup 3 — Users Interviewed and Users Not Interviewed: We compared the questionnaire responses from the 30 teachers we interviewed to those of the 98 teachers we did not interview. The results of our Chi-square tests indicate teachers who were interviewed were not statistically different from those who were not interviewed. Table 20 contains the results of these analyses. Table 20 Comparisons of Responses of Teachers Interviewed and Not Interviewed (Questionnaire Data) Variable (if a 12 p Grade Taught 10 127 6.831 .741 Years Teaching 32 126 41.281 .126 Educational Level 3 124 2. 138 .544 Type of Problem 2 127 1.521 .467 Who Responsible for Developing Interventions 9 128 8.382 .496 Teacher Role in Developing Intervention 6 128 2.492 .869 Teacher Agreement with Plan 3 126 3.821 .281 Extent to which problem was resolved 3 128 4.313 .230 Satisfaction 3 128 4.403 .221 Refer More Students 3 128 1.230 .746 Recommend to Other Teachers 3 128 4.677 .197 Additional Analysis — Nonresponders and Responders Before going on to our second question, we want to present the results of one additional analysis, the follow-up of nonresponders. The reader will recall that we obtained high response rates in ten of the thirteen schools in which we conducted our initial questionnaire research, and according to Babbie (1989) our 72% overall response 71 rate was very good. However, because we did not obtain the 80% response rate recommended by Lockhart (1984), we conducted a follow-up to determine whether our sample was representative of the teachers in the district, asking nonresponders about Use, Recency of Use, and Satisfaction. We asked nonresponders to tell us first, whether or not they had ever used the teams, second, when they had last used the teams, and finally, whether or not they had been satisfied with their experiences with the teams. We then compared their responses to questionnaire data from responders (Questionnaire Items 7, 10, and 22). For all three questions, nonresponders were consistent with responders. All nine nonresponders (100%) said they had used the teams, which is consistent with our high (83%) use of teams by responders. All nine nonresponders (100%) also said they used the teams within the previous 18 months, which is consistent with our Users rate of 99.2%. Finally, when we asked nonresponders whether they had been satisfied with their experiences with the team, only two of the nine nonresponders (22%) said they were satisfied with their experiences. This is commensurate with teacher reports of satisfaction on our questionnaire, where less than one-quarter of the 128 responders (21.9%) indicated they had been satisfied with their experiences. It should be noted the interview data regarding satisfaction presents an even bleaker picture. In examining the responses on the questionnaires of the teachers we interviewed, 15 of the 30 (50%) indicated they were satisfied with their experiences with the team. However, during the interviews, 13 of these 15 teachers said they were actually dissatisfied with the teams, but had not wanted to say so on the questionnaires. They provided a variety of reasons for not wanting to indicate their dissatisfaction in the written format of the questionnaire, ranging fi'om not wanting to disparage team 72 members, to concerns about possible repercussions because of negative responses. We do not know whether the teachers who were not interviewed would change their responses when a different response format was presented them. Because we do not know that, we do not know whether teachers who were interviewed were, in fact, less satisfied than teachers who were not interviewed. One key finding is that the methodology (questionnaire v. interview) had quite an impact on respondents. Future research is needed to explore this question more fully. Question One: Summary To summarize data about our first research question: “Who uses the teams?” we found the majority of the teachers in the sample (83%) use the teams. Of the 128 Users, virtually all (99.2%) who had ever used the teams used them on an ongoing basis. Analyses indicated that subgroups were comparable, that is, Never Users were representative of Users when first year teachers were removed from the data set. Teachers Interviewed were representative of those Not Interviewed, and Nonresponders were representative of Responders. The most remarkable finding to emerge from the analyses is that 100% of the ten first year teachers who completed and returned the questionnaire, indicated they had never used the teams. We do not know why this is the case, and consider it an area that should receive additional examination. Characteristics of Referrals To this point, we have analyzed demographic data about teachers, because teachers and their perceptions of prereferral intervention teams are the focus of this study. Although beyond the purview of this study, we also looked at demographic data about students who teachers refer and the characteristics of the referrals. Specifically, we 73 looked at 1) numbers and types of problems teachers referred, 2) gender, retention, and referral history of students referred, and 3) number and types of interventions teachers implemented prior to referring students to the team. For those who are interested, these data are presented in Appendix G. Team Membership and Referral Procedures Having addressed the demographic data, we now intend to move on to research questions 2, 3 and 4. Before doing so, we need to present three key pieces of information relative to the prereferral meeting: 1) team membership, 2) length of the meetings, and 3) teachers’ expectations of the team. These are important to thoroughly understanding data presented in Questions 2, 3, and 4. Team Membership Team membership is an area that has been fiilly described in the literature. It varies according to each specific model, ranging from specialist-led models (Fuchs, 1987; Graden et al., I985b; Rosenfield, 1992) to teacher-led models (Chalfant & Pysh, 1989; Chalfant et al., 1979; Pugach & Johnson, 1988). All models emphasize the importance and role of the referring teacher on the team. The majority of prereferral programs discussed in the literature follow the specialist-led models. They define core team membership as including the referring teacher, principal, school psychologist, school social workers, and special education teacher consultants (Bahr, 1994; Bahr, Whitten, & Dieker, 1999). The teachers we interviewed did not consider themselves core team members. Instead, they referred to the itinerant special education specialists (psychologist, social worker, and teacher consultant) as the ““team,” with school-based personnel (principal, 74 teachers, counselor, nurse, etc.) in a secondary role. While questionnaire data about who attended meetings (Table 21) paralleled results found in other studies (Bahr, 1994), the fact that teachers generally did not consider themselves members of the teams is inauspicious. For whatever reason, they do not view themselves as members of team deliberation and decision making. Instead, they view themselves as outsiders. This goes against everything we know about effective prereferral intervention programs. However, because the literature that addresses team membership does not provide information that clearly defines teachers’ roles or how they perceive their roles, we do not know whether our teachers differ from teachers in other programs. In other words, we do not know whether general education teachers who refer students to these teams generally view themselves as outsiders rather than integral members of the team. If they do, it would certainly have implications for how they respond in practice and for firture data collection and data interpretation. Because we do not know the answer, we can only caution the reader to keep in mind when interpreting questionnaire data, that when we asked teachers about their perceptions of these teams, the people to whom they were referring were the School Psychologist, School Social Worker, and Teacher Consultant. Table 21 Teacher Reports of Person_s in Attendance at Prerefen'al Meetings (Questionnaire Data) Frgueng Percent Referring Teacher 126 98.4 School Psychologist 104 81 .3 Special Education/Teacher Consultant 104 81.3 School Social Worker 100 78.1 Principal 76 59.4 Other (HI/ VI/SLI teachers. nurse. counselor. family members) 35 27.3 Parents 35 27.3 Other GE Teachers 34 26.6 75 Length of the Meeting We assumed programs that followed the problem solving processes would include the time necessary to implement the procedures. We had not even considered the reality. Teachers we interviewed said meetings were scheduled to last one half hour, every other week, before school. During these meetings, two, or more often, three, teachers were scheduled to present their referral. The third teacher was likely to have his or her time used up by the other two, and had to leave for class before having the opportunity to complete the discussion of their student. Apparently, the expectation is that the problem solving process will be completed within this ten minute period, a time period we doubt could be used to do more than scan the cumulative folder. So when we present the results of the interviews relative to what happened at meetings, it is important to keep in mind that the meetings lasted only ten minutes. Expectations Teachers view prereferral intervention as a necessary and preliminary requirement to be met before acquiring their ultimate goal — referral to and placement in special education. Twenty-eight of the teachers interviewed (93%) said they referred students they believed to be in need of special education, and wanted the outcome of the prereferral to be placement in special education. This is an important consideration when we examine teachers’ perceptions of and satisfaction with their experiences, because it would suggest any outcome short of placement in special education would result in their dissatisfaction. It is also important to understand teachers’ intentions because it raises questions about their potential for involvement in processes that would maintain the student within general education. 76 With these three factors in mind, 1) teachers do not consider themselves members of the teams, 2) meetings last ten minutes, and 3) teachers want the outcomes of the meetings to be placement in special education, let us move on to the results of the analyses of our remaining questions. Question Two: Problem Solving and Communication In this section we address information related to our second question: “What are teachers’ perceptions of the problem solving processes of prereferral intervention teams and of the communication that facilitates problem solving?” In this section, we address three major goals: First, we examine what teachers think about each of the four components of the problem solving model: 1) problem identification and clarification, 2) development of plans and interventions, 3) development of procedures for implementation of intervention plans and 4) development of procedures for evaluation. Second, we present information about teachers’ perceptions of their roles in the programs. Finally, we discuss the communication they describe occurring throughout the process, and the extent to which they said their experiences were collaborative. Before we continue, several words and terms appeared to have been interpreted in different ways by different teachers. This was discovered during the interview, when teachers described occurrences that were at odds with what we expected. For example, we already discussed that teachers’ definitions of the word “team” were different than those used in the prereferral intervention literature. We also found that the term “collaboration” was variously interpreted as meaning 1) mutual co-construction, 2) having a discussion, 3) having the team agree with teachers that students needed evaluation, and 4) speaking to each other politely. Because these language subtleties 77 appeared throughout the questionnaire and interviews, in the remaining sections, when we discuss our findings, we will repeatedly point out instances in which we believe language to have been different from what we expected, because they have an impact on how we interpret and reconcile findings from questionnaire and interview data. It should be noted, though, that if we had not used the methodology we chose, both questionnaire ap_d_ interview, we would not have been aware that teachers were interpreting questionnaire items differently than the way in which we had intended. Problem Identification and Clarification The first component of the problem solving model we asked teachers about involves problem identification and clarification. As described in the literature, this is a process in which multiple sources of information are examined and considered. Team discussion at this stage should lead to a decision regarding whether additional data should be collected and the problem reconsidered. We know careful consideration of information at this stage often results in redefinition of the presented problem (Johnson & Pugach, 1991), an outcome that logically allows for more targeted successful interventipns. However, thorough analysis of information and additional data gathering was not what teachers in our interviews described. Instead, they described team members conducting a cursory review of data teachers presented. Types of information gathered: We asked teachers we interviewed what types of information team members gathered before the meeting. We asked 1) whether or not anyone contacted them before the meeting, and if so, 2) what kind of information was gathered before the meeting. Most of the thirty teachers we interviewed (80%) said no one from the team contacted them, reviewed the students’ file, observed the student, 78 reviewed work samples, or in any other way gathered information before the prereferral meeting. The remaining twenty percent of the teachers (it = 6) reported team members made incidental contact with them or their student, or conducted only a brief observation (five to ten minutes). Types of information reviewed at the meeting: We also asked teachers we interviewed what types of information were reviewed at the meeting. Essentially, they told us the only data gathering process team members engaged in was listening to a verbal presentation by the teacher, and then asking a few questions. The interview data supports this because ninety-seven percent of the teachers (n = 29) reported giving a verbal presentation of the types of problems the student was experiencing and what they had done to try to resolve the problem. Either during or following their verbal report, teachers said team members might ask them questions. Typically, these took the format of “Have you tried this (specific intervention)?” or “Have you tried that (specific intervention)?” The twelve teachers we interviewed (40%) who described being asked if they had tried particular interventions all said they already had, although four mentioned that when they were less experienced, the team provided them with useful suggestions. This corresponds with the literature on prereferral intervention that indicates that 90% of teachers report teams asking them to attempt interventions they already tried (Inman & Tollefson, 1988). The only actual data reviewed at the meeting was information from the cumulative file, which teachers said typically consisted of attendance records, discipline referrals, medical information, grades, retention history, and results of any standardized tests the student had taken. Eighty-seven percent of the teachers interviewed (n = 26) said 79 they brought the students’ cumulative folder to the meeting, and team members reviewed what was in that file. Thirty percent of the teachers we interviewed (n = 9) brought work samples, although three of them said the team members did not look at them. Although many of the teachers said the outcome of the meeting was informal testing to see if the student might require a complete assessment for special education eligibility, in no cases (0%) were the results of those assessments used to develop interventions for implementation in the classroom. In fact, teachers said the only outcome of informal testing was either formal testing for special education, or a note in their mailbox saying “results of screening indicate the student would not qualify with additional testing.” They also said when students qualified for special education, teachers still were not given strategies for how to work with them in the general education classrooms. Table 22 Mcher Reports of Types and Sources of Information Reviewed at Meetings (In_terview Dm) Teacher Team Member Frequencv Percen_t Mum Picagu Before Meeting Cumulative Folder 26 86.7 -- -- Work Samples 9 30.0 -- - Verbal Report 29 96.7 - - "Formal" Observation (15 rrrinutes) -- - 2 6.7 Informal or Incidental Observation -- - 2 6.7 Previous Evaluations (for Special Education) -- -- 2 6.7 After Meeting - for 1n_terven_tion Developmen_t Teacher Report - -- 1 3.3 Observation -- -- 1 3.3 80 Teachers’ reports suggest a process where the only information considered is that which the teacher brings to the meeting, which, in most cases, is the cumulative folder only. As such then, team members primarily considered data from the cumulative files such as attendance, retention history, and the results of any standardized testing, as well as any other information presented by the teacher. Essentially, this means the teacher is responsible for the amount and type of information that is considered. Over half of the teachers we interviewed (57%), particularly the less experienced ones, were uncomfortable with this procedure. They expressed concerns about not knowing what to present to the teams, and not having the information that was available, looked at. They came in and asked for his file and had it for a while and brought it back, because I remember having to get it out. Not work samples, just his file. I wish they would tell me what they want. I’d give it to them if I knew. The student and his dad told me that he was in special ed somewhere else, and he gave me the paperwork and I gave it to them, but they didn’t look at it. I tried to give them work samples but they didn’t compare them to test scores. First thing they say is “Let me see the attendance card.” They usually don’t even look past that. The student spoke Spanish, they didn’t. They said it was language, but it isn’t. I speak Spanish, and it’s the same problem there that it is in English. But they wouldn’t let me interpret for them, and they couldn’t talk to her. Although teachers did not object to this explicitly, another concern we raise is that in most cases, teams do not gather any additional data to corroborate or repudiate teachers’ observations. Since the only data considered is what teachers bring to the meeting, the plans being developed might overlook subtle but important factors that, given their other responsibilities, may be difficult for teachers to notice. 81 Problem identification - summary: Before moving on to discuss plan development, let us summarize the data about problem identification and clarification. First, other than the cumulative file, there is no clear understanding about the data set that all teachers should bring to the meeting. Second, there is essentially no additional data gathered by team members before the meeting. Third, any data that are gathered after the meeting are largely to determine whether the student is, or might be, eligible for special education, not to provide additional information for problem solving. Finally, the available meeting to discuss a given child, ten minutes, provides virtually no time to discuss problems in a way that might lead to problem clarification. Given that we know effective use of problem identification procedures can result in significant changes in how problems are viewed, and therefore resolved (Johnson & Pugach, 1991; Zins & Ponti, 1996), the lack of application at this step is troubling. Plan Development The second component of problem solving we asked teachers about was plan development. As with problem identification, the literature provides a series of steps to ensure the development of effective plans. Once the problem has been adequately defined, effective plan development includes 1) generating multiple possible strategies, 2) discussing the potential problems of each before arriving at a plan, 3) developing a written plan that outlines procedures for who will be responsible for specific components of the plan, and 4) provisions for evaluation. The questions discussed in this section were designed to determine whether and to what extent, the aspects of plan development listed above were present in the process. 82 Components of the process: As described by teachers we interviewed, the steps for effective plan development were even less likely to have been used than those in the problem identification component. With respect to plan development, teachers reported that at the end of the “data-gathering” phase of the meeting, a team member announced a “plan - of sorts.” Basically, this consisted of announcing what would happen next, statements which teachers typically considered ““the plan.” None of the 30 teachers we interviewed (0%) said different strategies were generated during the meeting. Only two of these 30 teachers (6.7%) said a variety of strategies were discussed before arriving at a plan. None of the 30 teachers(0%) had seen a written plan, and in none of the 30 cases (0%) were specific provisions made for evaluation. These data are presented in Table 23. 83 Table 23 Teacher Reports of Use of Componerus of Plan Development (Interview Daaa) F uen ' Percent Generating different strategies 0 0.0 Discussing different interventions 2 6.7 Written Plan 0 0.0 Specific Definition of Roles 0 0.0 Specific Definition of Responsibilities 0 0.0 Before we continue, it is important to note that the term “plan” is used differently by teachers we interviewed than as it is defined in the literature on prereferral intervention. When teachers talk about plans, the usually mean “a plan to conduct additional assessment.” If they are talking about a plan to work with a student, they usually mean one or two verbal recommendations for loosely defined strategies. As “plan” in the literature means a written outcome emanating from procedures that involve problem identification, plan development, implementation, and evaluation, and as teachers report these key elements are missing, we have concluded the word “plan” as used in the literature cannot be used to discuss the decisions made at the meeting. We decided from this point on to use the word plan (italicized) when referring to the plans teachers referred to so the reader will not confuse what the teams actually do with the well-developed plans the literature describes. Plan Development In order to consider how plans were developed, we need to first examine the nature of the plans themselves. We asked the teachers we interviewed to describe, in detail, the recommendations made by the team. Their responses are listed in Table 24. 84 Table 24 Plans Develoned in the Prereferral Process (Interview Data) Fraquencv Percent Testing formal or informal 17 56.6 ADD evaluations (teacher checklist) 3 10.0 Behavior Plan 4 13.3 Improve Attendance (no plan) 3 10.0 Wait a Year (no plan) 2 6.7 Don‘t Know (I‘m waiting (six months) for them to tell me) 1 3.3 When we look at the actual content of the plans, we find two-thirds of the students referred to the team (66.7%) were subsequently referred for additional evaluation. Another twenty percent (n = 6) were remanded to the teacher with no specific suggestions beyond “wait a year,” or “improve attendance.” None of the thirty teachers (0%) described any academic interventions being recommended. Only four of the thirty teachers we interviewed (13.3%) reported any semblance of strategies or recommendations. In all four of these cases (100%), the interventions recommended addressed behavior problems. Two of these four teachers said the interventions were quite brief, either “praise the student more,” or ““modify your existing time-out plan so the student does not have materials.” The other two teachers described interventions that were slightly more complex. One was a system for ensuring supervision in the case of a student who regularly ran away from the class. The other involved a token reinforcement system. The literature suggests the majority of interventions developed by prereferral intervention teams are designed to address behavioral concerns (Chalfant et al., 1991; Pugach & Johnson, 1995). As such, our finding is supported by the literature. It should be emphasized, however, that m of the four plans developed constitute prereferral intervention plans according to how these are defined in the literature. 85 Teachers’ roles in plan development: On the questionnaire, we asked teachers a series of questions designed to examine their role in plan development, including who was most responsible for developing plans, and what their own role was in developing plans. Summarized in Table 25, these data largely indicate teachers (46.9%) perceived themselves as most responsible for plan development. Table 25 Teacher Reports of Who was Most Resppnsible for Developing Iruerventions (Questionnaire Data) Frequency Percent Referring Teacher 60 46.9 Teacher Consultant 31 24.2 Entire Group 19 14.8 School Psychologist 5 3 .9 School Social Worker 5 3.9 Other General Education Teachers 2 1.6 Principal 3 2.3 Other (Parents. Nurse. Counselor) 3 2.3 We are not entirely sure what this means. The most optimistic interpretation would be that teachers were well represented within the meetings and had a lot of say in making decisions, a factor the literature says is important in ensuring that good communication occurs during problem solving (Adelman & Taylor, 1998; Graden et al., 1985b; Knoff et al., 1995a; Knoff et al., 1995b). Unfortunately, this is not a likely interpretation considering everything we know about the process in this study. Additionally, that they were responsible for plan development in a venture that is supposed to be collaborative might also mean teachers do not feel that they have been provided with the necessary support. Further evidence for the latter interpretation can be found in teachers’ descriptions of the procedures followed during plan development, where they report 1) different strategies were not generated, 2) different strategies were 86 not discussed, 3) no written plan was produced, and 4) no provisions were made for implementation or evaluation. We asked some of the teachers we interviewed to help clarify what teachers might have been thinking as they answered this questionnaire item. We received two possible responses. First, it appears that some teachers who indicated they were most responsible for plan development on the questionnaire were referring to interventions they developed and implemented before going to the prereferral team. Second, others who indicated they were most responsible for plan development on the questionnaire were referring to interventions they developed and implemented later, because, during the meeting about their student, the team made no recommendations. Data we present later says teachers are largely dissatisfied with these teams, so we might infer responsibility for plan development is not a role teachers want. However, in order to determine what is meant by teachers saying they have primary responsibility for intervention development, additional data are needed. On the questionnaires, we also asked teachers to specify what their role was in the development of interventions. The response choices and results are listed in Table 26. While a large portion of teachers (34.4%) said their role consisted of presenting information and being told what to do, over one-third (35.2%) said they collaborated in plan development. This figure of 35% seemed high considering their perceptions of the plan development process and their role in problem identification. 87 Table 26 Teacher Reports of their Own Role in the Development of lnten'engons (Question_naire Dat_a) Frgueng' Percent No role 8 6.3 Presented information was told what to do 44 34.4 Others developed interventions. 1 selected the "best" 2 1.6 Collaborated 45 35.2 Assumed leadership role 16 12.5 Missing 13 10.2 However, interview data again presents a different picture. When teachers were given a working definition of collaborative and hierarchical approaches, and asked to describe where their teams fell on a continuum of collaborative to hierarchical, “expert” models, the majority (90%) said the teams were not collaborative (see Table 27). When we compare that to the literature that emphasizes the need for teachers being responsible for decision making, our results are not promising. Table 27 TLacher Reports of their Team Relative to Collaboration or Expert Modelsjlnterview Dat_a) F uen ' Percent Very Collaborative 1 3.3 Somewhat Collaborative 2 6.7 Not Very Collaborative 8 26.7 Not Collaborative at A11 - Experts Prescribe 19 63.3 We assume our use of a different format, when we clarified what we meant by “collaborative” on the interview, caused the differences in teacher reports of their role in plan development on the questionnaire and interview. One teacher we interviewed who said her team was quite prescriptive was asked about her questionnaire response indicating her role was collaborative. She said, 88 The team we had before was incredible. You wouldn’t believe how rude they were. One of them spit at a teacher once. This years’ team isn’t that way. They listen to you, and they usually don’t interrupt. They’re polite. 5’ 6‘ Considering that teachers regard the word plan as meaning either “testing, no strategies,” or “simple, generic strategies,” it is not surprising they define the word “collaborative” as “good manners” or “communication,” rather than mutual co- construction of solutions to clearly defined problems. Plan development — summary: Before we discuss the next step in the problem solving process, let us summarize the data about plan development. First, teachers did not perceive any use of procedures for plan development as these are described in the literature on prereferral intervention. That is, they said they did not generate or discuss multiple strategies, and there were no written plans outlining procedures for implementation or evaluation. Second, the majority of the plans that did result from the meetings involved sending the student for further evaluation. In the remaining cases, there were either no specific strategies recommended, or, where they were, plans were limited in scope and poorly designed. Third, questionnaire data tells us teachers view themselves as primarily responsible for plan development, which is seemingly contradicted by interview data that tells us team members largely dictate plans. However, an analysis of the data suggests teachers are likely talking about two entirely different things. We need more information before we can determine what teachers’ actual roles are in plan development. Agreement with the Plan On the questionnaires, we asked teachers the extent to which they agreed with the plan that was developed. Their responses, depicted in Table 28, indicated only one-third (32.8%) agreed with the plans. This is odd, given that questionnaire data responses 89 indicated 46.9% of teachers said they were primarily responsible for developing the plan (see Table 25). Table 28 T_e_a_cher Reports of Agreement with the Plan (Questionnaire Data) Frequencv Percent Agreed Completely 42 32.8 Agreed Partially 50 39.1 Disagreed Somewhat 17 13.3 Disagreed Completely 17 13.3 Missing 2 1.6 Again, we find what appears to be a discrepancy. However, in looking at data in other sections of the questionnaire and interview, it becomes apparent that teachers are, once again, responding to different questions. It appears likely that when we asked: “Who was the person most responsible for developing interventions?” (Questionnaire Item 14), teachers were interpreting the question in many ways. They may have been responding in terms of interventions they developed prior to initiating a referral. Or, they may also have been referring to interventions they developed when the team provided them with no additional suggestions. However, when asked: “To what extent did you agree with the plan that was developed?” (Questionnaire Item 16), they may have been referring to whether or not the student was evaluated for special education, which we know was their primary objective in referring the student. So, it is not surprising that teachers did not agree with plans as they were described in interviews, given that 1) the stated major goal of 93% of teachers is for the student to be placed in special education, 2) over 40% described having minimal involvement in intervention development (see Table 26). 90 Procedures for Evaluation By and large, teachers we interviewed indicated that in the few cases where there were plans, these were poorly developed, because they were not data-based, they were not recorded, and they had no procedures for implementation. It should come as no surprise, then, that few teachers we interviewed were aware of any procedures for evaluation of the outcomes of these plans. The few that were, were those who were told that testing could not occur until a students’ attendance improved, and that attendance data could be reviewed in another six months. However, there were no specific criteria for what constituted “improved” attendance, and there were no specific strategies developed for how to improve attendance. Finally, before moving on. teachers considered it a plan when students were to be tested. These plans, although designed to gather additional information about the student, were ends in and of themselves, rather than part of problem identification. Teachers we interviewed said when testing was informal (that is, it did not lead to an eligibility determination), the actual results were not discussed with them, and data obtained were not used to develop any specific strategies for the teacher to work with the students. Further, there were no plans developed when formal testing indicated the student was not eligible for special education. So while additional data were collected about the student, they were not used to provide teachers with information that might help them work with the student in their classes. Question Two: Summary When we look at the specific components of the problem solving model and address how these were implemented, prereferral intervention plans as defined in the 91 literature do not exist as an outcome of the prereferral intervention programs we studied. Problem identification consists of teachers presenting verbal reports to teams, the team members asking questions, and occasionally reviewing cumulative files and work samples, all during a ten minute segment. There is essentially no additional data gathered by team members in order to develop interventions that might prevent referral to special education. Plan development consists primarily of a team member pronouncing what will happen after the meeting. As teachers describe it, the word “plan” is synonymous with 1) referral to special education, 2) referral to the Teacher Consultant or medical doctor for more information, 3) simplistic strategies such as “wait and see,” or “improve attendance,” or 4) one or two general/ generic strategies. Generally, there were no procedures developed for follow-up, either evaluating the outcomes of the strategies recommended or to communicate the findings of evaluations to the teachers. The only exception was the ill-defined “improved attendance” recommendation. Recognizing that there is no similarity to a plan as described in the literature, it appears that, other than testing, the best teachers can hope for in terms of these prereferral intervention programs is to be provided with a strategy or two. Unfortunately, our findings are similar to those of F lugum and Reschly ( 1994) and Fuchs (1987). These studies indicate the majority of plans developed by prereferral intervention programs are not data-based, and are generally lacking the basic components of the problem solving processes, or, when the components are there, they are poorly designed. Involvement and Demographic Patterns Before we proceed to discuss question three, we want to discuss some correlational studies we conducted. We decided to look at questionnaire data in terms of 92 what variables were predictive of whether teachers indicated they were involved in the development of interventions. Specifically, we wanted to know whether teacher statements that they were involved in the development of interventions were predicted by demographic variables (years of experience, educational level, or grade taught). We defined involvement in the development of interventions as the two response choices on item 15 of the questionnaire where teachers responded that they either collaborated or assumed a leadership role (See Table 29). By this definition, 53 percent of the teachers who responded to this question indicated they were involved in the development of interventions. Table 29 Igachcr Reports of Involvement in Plan Development (Questionnaire Data) Fregueng' Percent Valid Percent No 54 42.2 46.9 Yes 61 47.7 53.0 Missing 13 10.2 -- Using stepwise logistic regression procedures, we analyzed the relationship between demographic data (years taught, educational level, and grade taught) and whether or not teachers described themselves as having been involved in the development of interventions. The likelihood ratio chi-square was statistically significant [LR x2 (2) = 6.82, p = .03], indicating a model for the relationship that contained two explanatory variables (years of teaching and educational level). Table 30 summarizes the contribution of these two explanatory (independent) variables to the prediction of collaboration. 93 Table 30 Contribution of Demographics LYears Teaching, Educational Level) to Involvemeat (Questionflre Deg) Parameter Estimate SEM 1; 9 Odds Ratio Intercept 0.49 0.69 0.52 0.47 -- Years of Teaching 0.78 0.31 6.26 0.01 0.46 Educational Level 0.69 0.44 2.47 0. 12 1.99 In other words, the more years of teaching experience a teacher has, the more likely the teacher is to have indicated they were involved in the development of interventions. And, the higher the teachers’ educational level, the more likely the teacher is to have indicated they were involved in the development of interventions. The grade taught does not predict teachers’ perceptions of their level of involvement. Additionally, there is an interaction between years of teaching and involvement, depicted in Table 31, that tells us that the combination of increased years of teaching and increased education maximizes the likelihood that a teacher will have indicated they were involved in the development of interventions. In other words, teachers whose years of teaching are high but do not increase in educational level (possibly those with lifelong certification), or teachers whose educational level is high but years of experience are lower (for example those with Master’s degrees who have left and returned to teaching) are less likely to say they collaborated than are teachers with both higher levels of education and years of teaching. It may be teachers with the combination of higher education and experience are more confident than other teachers. It may be their knowledge base is broader, or they perceive themselves specialists in their area, and as such, are more comfortable working with other specialists. 94 Table 31 Interaction Between Years of Teaching and lnvolvemenijucstionniire Dala) Not Involved Involved Years Teaching 1 (0-5 years) Frequency 13 9 Percent 10.32 7.14 2 (6-20 years) Frequency 27 31 Percent 21.43 24.60 3 (21+ years) Frequency 16 30 Percent 12.7 23.81 Education BA-BA+ Frequency 26 33 Percent 20.97 26.61 MA — MA+ Frequency 30 35 Percent 24.19 28.23 The literature on consultation was equivocal about who uses consultation services. One study said novice teachers are more likely to use the services because they know they need help (Mann, 1973). Another said more experienced teachers are more likely to use services because they are not concerned about how they are perceived when they ask for help (Stenger et al., 1992). Our data suggest that both those results are valid. While the majority of teachers in the sample used the teams, there are differences related to experience and education that suggest novice and expert teachers’ roles in the consultation processes are significantly different. Question Three: Intervention Implementation In this section we address information relative to our third question: “What are teachers’ perceptions of their own roles on these teams relative to the implementation of 95 prereferral interventions?” Keeping in mind the plans were not developed using the procedures outlined in the literature, and were poorly designed, we will discuss who was responsible for implementing interventions and whether or not they were implemented. We will also describe teachers’ perceptions of the specific strategies they were asked to implement. Responsibility for Intervention Implementation According to the teachers we interviewed, responsibility for specific components of plans was assumed — that is, teachers “knew” that students who were being tested, either formally or informally, were being seen by a team member, although not necessarily which one. They knew that “try this (specific strategy),” or “wait a year,” meant they were responsible for resolving the problem. However, when we looked at the questionnaire data, we ran into the same problems with language in looking at intervention implementation that we did in problem identification and plan development. That is, questionnaire data are flawed because of how terms were defined. When we developed our instruments, we assumed, going in, that each student would exit the meeting with a “real plan,” with goals to be implemented. But having looked at problem identification and plan development we know a) plans of the kind described in the literature were not developed, and b) for the majority of the students, teachers were given no explicit strategies (66.7 % referred for additional evaluation, 20.0% had no strategies recommended, 13.3% had limited strategies recommended). So while we have questionnaire data about who was primarily responsible for implementing the interventions, and whether or not they were implemented, we do not know whether questionnaire responses were related to interventions teachers developed and 96 implemented independent of the team. Consequently, the only data it makes sense to look at in this section is the data we obtained from the four teachers from the 30 we interviewed who specified they were given plans to implement. So although these are unlike the exemplar definitions we have, and although we recognize these data, coming from such a small sample (n = 4), are very limited, we simply report the data here in order to illustrate strategy implementation. Implementation of Strategies We asked the four teachers we interviewed who reported they were responsible for actually implementing strategies prescribed by the teams to describe their experiences. The presenting problem, “plan,” and implementation are described in Table 32. Keeping in mind that our sample size is only the four teachers who were given strategies to implement, three-quarters of the teachers implemented the strategies. Of these, half the teachers ( 50%) said the strategies were fully implemented, one-quarter (25%) described implementation that was inconsistent at best, and one-quarter (25%) did not implement the strategy. 97 Table 32 Strategies Recommended by Teams and Implemented vaeaLchers (In_terview Data) Presenting Problem Plan/ Strategy Implementation Resolution 1) Oppositiona1* Praise the student more None Problems continued 2) Oppositional“ Change existing plan from Full Improved for “awhile." time-out with materials to then regressed time-out without materials 3) Running away and Supervision for remaining Full Sent to self-contained hiding students while teacher class for students with addresses runaway behavior problems 4) Oppositional“ Points - token reinforcement Intermittent No change in behavior system after three months *Hits other students. acts "mean." throws things. does not follow teacher directions or requests. Teachers’ Perceptions of Strategies When we talked to the four teachers who reported having been given strategies to implement, we asked what they thought of the interventions. Relative to the cases presented in Table 32, in the first case, the teacher was incensed at the suggestion that she praise the student more. She said the students’ behavior was not deserving of praise, and instead, implemented a negative reinforcement system. The students’ behavior did not change. In the second case the teacher implemented the strategies, but only because “I thought they were so inappropriate I wanted to prove they didn’t work.” The teacher reports the intervention was effective for a short period of time before the student stopped responding. In the third case, the strategies were changed daily as the student continued to run away. The teacher said she was grateful for the teams’ support, and fully implemented whatever was asked of her, saying, “I was desperate, I didn’t know what to do, and nothing I had done worked, so I did whatever they said.” The strategies were ineffective, and the student was transferred to a special program in another school. With 98 the fourth student, implementation was inconsistent. It was clear from the teachers’ description that her training in using the token system was only procedural, and that she did not understand how to use the strategy to shape the student’s behavior. It should be noted that three of these four teachers were relatively low in experience and education. The other teacher was highly educated and experienced. However, this teacher was the teacher of the student who was ultimately placed in a specialized classroom. We know the literature on intervention implementation says that when interventions are developed with high levels of adherence to the components of the problem solving process, they are usually implemented accurately (Noell & Witt, 1999). Based on descriptions provided by our teachers, it is not surprising that interventions were not implemented accurately, given the lack of quality and adherence to the components of the problem solving processes. Moreover, research indicates that low levels of adherence, such as those described by the teachers in this district, are as ineffective as no interventions at all (Kovaleski, Gickling, & Morrow, 1999; Noell & Witt, 1999). Question Three: Summary To summarize our data relative to our third question: “What are teachers’ perceptions of their own roles on these teams relative to the implementation of prereferral interventions?” we know that with no written plan, there was no formal assignation of responsibility. The person responsible for implementation was assumed, based on the content of the plan. For example, if the plan called for additional testing, then one of the specialists was responsible, while if other plans involved “waiting a year,” the teacher was responsible. We excluded questionnaire data from our analyses because we could not 99 determine what teachers meant when they responded to the questionnaire item “Who was most responsible for implementing the intervention?” As such then, we considered only data from the four teachers we interviewed who were given strategies to implement. These data indicated that while strategies recommended were poorly designed, three of the four teachers attempted to implement them. Further, while their implementation was ineffective, and from their descriptions, inconsistently applied, it was clear from teachers’ statements they did not understand the conceptual foundations for the strategies, and at best, had only been given procedural information about how to implement them. Even though teachers’ goals of having the student receive special education services may have impacted their willingness to implement interventions, their having been given such poorly planned interventions to implement would logically preclude success. Under these circumstances, the only logical or responsible thing for teachers to do is to try to ensure students receive special education services. Goals of Prereferral Intervention Programs Revisited Before we look at our final question, we thought it would be helpful to review the purposes of prereferral intervention programs described in the literature and compare them to the information we have discussed so far. The first major goal of prereferral intervention is to provide classroom support so students’ problems do not become severe enough to warrant referral to special education. That a) 28 of the 30 teachers we interviewed (93.3%) said they believed the particular student they referred needed special education, b) six of the 30 teachers (20%) were given no suggestions at all, and c) all (100%) of teachers we interviewed said when testing did not result in eligibility they were not given any suggestions, it appears this goal is not being met. The second major IOO goal of prereferral intervention programs is to provide teachers with professional development skills that would allow them to address ongoing and future problems. However, 40% of the teachers we interviewed said they were provided with strategies they had already tried, and as such, learned no new interventions or skills. In those few cases where teachers were actually “instructed” in how to implement strategies (recall these represent only three cases), they were given just enough information to implement the given strategy, but no information about the concepts underlying the strategy that could allow them to be flexible in implementing the strategy. Furthermore, because none of the teachers received any substantive training, they were hampered in their ability to generalize any skills they aligm learn. These suggest the goals of professional development are not being met. Supports and Resources This section addresses information related to the fourth research question: What kinds of supports and resources do teachers say they need in order to make prereferral interventions succeed? When asked this, teachers pointed out that the teams usually did not provide them with interventions. However, they willingly described supports and resources that they felt would be useful to improve overall team firnctioning. These involved 1) time, 2) training and service delivery, and 3) more voice in decisions regarding special education eligibility. Time Although we did not ask them specifically about the time flame of their experiences, with respect to the issue of time, 87% percent of the teachers interviewed 10] (n = 26) said lack of time was a widespread problem. They voiced four major concerns in this regard: I) the wait before a referral, 2) the wait between the initial referral and the meeting, 3) the lack of time at the initial meeting, and 4) the amount of time team members spent in the building. Time to the referral: Six of the 30 teachers we interviewed (20%) expressed concern about the time it took to initiate a referral. Teachers refer students by signing up in a notebook in the office. Referrals are then heard in the order of sign-up. Some teachers said “the book” was not available to them until the middle of the school year. The names on the list they created as a substitute were placed at the end of the list when the “real book” was finally available. One teacher described what happened to a teacher who missed a meeting. “We’re usually given the note in the box the night before, and she missed the meeting. Her kid had to go back to the bottom of the list.” Time between the referral and meeting: Thirteen of the 30 teachers we interviewed (43%) wanted quicker responses to their referrals, saying it took too long to access the team. Typically, they said it took six to eight weeks from the time they “put the student’s name in the book” to the time of the first meeting, and there was rarely any contact prior to that meeting. Many teachers described waiting several months to have the meeting about their student. One said, “Our team works really fast, the time from when we put her in the book to the time of the meeting, was October to just now, (end of January). In some schools it takes the whole year.” One of the stated advantages of prereferral intervention programs is they provide immediate help to teachers. In informal conversation with district specialists, we were told the district’s prereferral intervention policies were designed to ensure the teacher 102 would be contacted within two school days to begin to identify and clarify the problem. According to the specialists, each team member is assigned to the building on a different day of the week. Each is expected to “check the book” on the day they are in the building and contact teachers that day to begin preliminary data gathering and problem analysis. Teacher reports indicate there is a wide gap between policies and actual practices, as none of the teachers reported this type of rapid response. Time at the initial meeting: Ten of the 30 teachers we interviewed (33%) also wanted more time at the initial meeting to discuss the student. They viewed the ten to fifteen minutes allocated as inadequate for conveying the student’s problems and describing their need. They said team members were rushed, did not pay enough attention to them, and often made decisions based on incomplete information. Further, 11 teachers (37%) said the time constraints often meant they had to leave meetings before they were complete, to attend to their classes. There was no method for communicating the outcome of the meeting in those cases. Teachers were not told of the team’s plan unless they were able to locate a team member at a later time. Time team members spent in the building: Additionally, 12 of the 30 teachers we interviewed (40%) wanted teams to spend more time in their building, because they wanted more immediate and lengthier evaluations. One teacher said when the outcome of the meeting was testing, backlogs were such that it sometimes required additional am for the team to begin the evaluations. Then, once evaluations began, three teachers ( 10%) said they did not think the typical brief (ten to twenty minutes) observations were adequate. One said, They need to know something about the child, more than what a test will show, or what’s written. They need to watch their facial expressions, and the off- 103 times, and the times when the child isn’t focused during academic times and at play times. That would give them more empathy with the child and with the teacher, and then they would be more willing to make more valid, non-textbook kinds of recommendations. These three teachers also expressed concerns that there was no way possible the limited amount of time spent testing could provide the examiners with adequate knowledge about the child. Somebody watched. . .well, generally, it’s ten to twenty minutes. Then the social worker talked to him for about ten minutes, or whatever... the diagnosis was he could talk, I suppose. Then Suzy Psychologist took him for, like, a half an hour or however long it takes to run him through the flip chart test or whatever it is. . . and then they came back with their results. Five teachers wanted all of the team members to observe each child referred, “Not just the one who was testing.” Several teachers (n = 12) stated the team members either were, or claimed to be, overworked, and thought reducing the team caseload would help. I’d like to see the team not feel so overwhelmed they don’t have time to talk with me or help the child. I’d like to see them help me understand what the problem is and what I can do effectively that would be of benefit to the child. Eight teachers said they felt strongly that increasing the time the team members spent working with teachers and students would improve the quality of the interventions recommended. They could give more meaningful suggestions - tailored to the teacher’s room and tailored to the student. Right now, the types of suggestions are more along the lines of, “Oh, you have a student who’s reading below grade level, well, have you tried partnering him with a better reader. . Well... I learned that in my first year of teacher school. It’s really general if they give you suggestions, those are like “Have you tried this, have you tried this, have you tried a chart on his desk with smiley faces?” They don’t take into account your other kids, what your personal style is, what the niles of your room are, what the kid reacts to. Instead of going through and just telling you to use a sticker chart for your child that seems to have trouble with impulsive behaviors, they could come in and say “You know, I’ve observed in your room for three days, and I’ve noticed that your kid has impulsive behaviors. But I’ve also noticed that the time he’s really impulsive is during your instructional times, and when you’re doing specific direct teaching. Have you 104 noticed that too? Well then let’s find a specific strategy to use during instructional times, but first, let’s identify when those are.” Then they would be able to say, “With this child, this result would probably work.” I don’t think that that always happens, I don’t think they get enough time to see each kid and enough time to know that child as an individual so suggestions become more generic. In summary, teachers wanted team members to respond quickly to referrals and to invest the time necessary to understand teachers’ concerns and students’ problems. They said they wanted team members to invest the time needed to ensure they provided quality evaluations and the services needed to ensure student success. Training and Service Delivery Nineteen of the teachers we interviewed (63.3%) also said they wanted changes in training and service delivery. These involved: 1) more substance in training in team procedures, 2) individualized contact with first-year teachers, 3) instruction in how to teach difficult-to-teach students, and 4) more creative and rigorous service delivery. Training in team procedures: The training teachers reported receiving was as follows: There was a brief introduction to the teams at staff meetings at the beginning of the school year. Lasting “about five minutes,” they generally were introduced to team members and told where and how to fill out the referral form (teacher name, student name, and whether the problem was academic or behavioral). They did not see this as necessary or adequate. With respect to training, thirteen teachers we interviewed (43.3%) wanted additional training in two areas: 1) the specific practices they needed to implement to ensure that the students they refer would be evaluated for special education eligibility and 2) written criteria for eligibility for special education services. Teachers viewed the teams as having a set of covert rules and regulations they follow in determining eligibility. The team meeting is simply a perfunctory required step 105 in the process. If teachers knew what the criteria were, and what policies and procedures were mandatory, they could achieve their goal of getting special education services for the student. Relative to interventions, teachers wanted specific instruction about what was expected of them that would ensure their student would be tested. They wanted concrete suggestions, and guidelines to follow as well as checklists they could complete in advance, then bring to the meeting to show the teams the student was “ready” for a special education evaluation. Relative to criteria for eligibility for special education, teachers wanted to understand the rules and procedures that were followed. Many teachers thought there were guidelines they could follow to ensure that students would receive special education services. They assumed they simply did not know these guidelines, and wanted these to be specifically taught. One teacher said that she found a manual that outlined district guidelines for special education eligibility, and made sure she used words from the manual when she described students, saying, “They take them if I do, they don’t if I don’t.” Another teacher said, We need to have guidelines. Teachers need to know exactly what it means to qualify. I can’t tell you the mean score to get a child in or out. I have no idea. Some of these things are secretive. They belong only in the psych department. They know how many percentage points the child must earn to qualify. Well, I’d like to know that too, and I would also like to know how the child failed - what the child failed... One teacher said: I don’t know what it takes to qualify. I think it takes room. If there’s room, then the child will qualify, and if there’s no room, then they’ll make up an excuse. Five of the 30 teachers we interviewed (17%) said the team certified “all” or “most” of the students they referred. One teacher commented, “I don’t mean it in a 106 disrespectful way, but I think I’ve learned a game. They test all the kids I send to them.” However, others perceived themselves as having been less successful, one said, “They’ve only certified a few of the students I’ve referred over the course of several years.” Two teachers we interviewed said they had never had a student that was evaluated for special education services. Teachers whose students were not certified were certain they had not presented their cases thoroughly or adequately. If their hypotheses are true, it may mean some teachers have a better understanding of special education procedures and eligibility criteria than do others. At the same time, it is not the stated intent that prereferral intervention programs are only for students who are being referred to special education. That teachers are so unknowing about this suggests a lack of awareness of the fimctions of the teams and a lack of communication with team members. It is disconcerting that they view the primary purpose of the team as determining which students should be referred for further testing, instead of as a resource whose primary fiinction is to address problems before they become serious enough to warrant referral to special education. When asked about this latter fimction, one teacher said, “Oh, yeah. I’d forgotten. That’s what it was supposed to be about. I wonder what happened to it.” Curiously, none of the teachers mentioned wanting substantive training in the problem solving processes, or in collaboration. In viewing the existing data, the primary reason for this seems to be that teachers were unaware that a problem solving process existed. When asked about the model, 24 teachers (80%) said they were not familiar with it. The six teachers (20%) who were familiar with the problem solving model said they learned about it during their academic training (undergraduate or graduate). One said: Actually, we learned about something like that when I was in college. It was called Child Study. One of my professors ran a very progressive 107 school on the East Coast. They used this Child Study method and we actually got to pick a child that was in our classroom where we were doing our internship and do a child study on them. But this was when I was totally ignorant of the prereferral intervention teams. I had no idea what actually happened in real schools, in the real world. Five of the six teachers who had ever heard of the components of the problem solving process said the teams in their buildings clearly did not use them. The only teacher who was aware of the problem solving model and said the teams used it was one of the most experienced teachers in the sample. This teacher was highly educated and had taught special education for twenty years prior to switching to general education. Training for first year teachers: Five of the 26 experienced teachers we interviewed (those with more than five years of experience) said first year teachers should get individualized and explicit training in using the teams, including instruction in how to ensure a student would be evaluated for special education. Seven of the teachers described incidentally having learned about the programs from other teachers. They said they wished they had learned sooner, faster, and better. Teachers remarked that while the district has a mentoring system for first year teachers, it is not explicit, and does not include any kind of formal instruction on prereferral intervention. Instruction in teaching difficult-to-teach students: Fourteen of the teachers we interviewed (46.6%) wanted instruction and training in how to address behavior and attention problems, and how to teach students who do not qualify for special education services, particularly students designated as Slow Learners. Where teachers described recommendations being made, those recommendations were usually generic, and did not provide the teacher with a specific plan to solve the problem or interventions they could realistically use while teaching an entire class of students. However, teachers said they had usually already tried strategies that were suggested by specialists and wanted new 108 ideas and information, “Real help. Things other teachers have done with similar kids.” They were particularly frustrated with being given such superficial strategies as “Reduce assignments,” and “Have the students complete every other problem.” One teacher remarked “How can this be an effective strategy when the student can’t complete any problems at all?” Service Delivery Teachers wanted team members to 1) observe teacher and student performance and make suggestions on how teachers could work more effectively with specific students, 2) provide in-class modeling, consultation and support, and 3) support teachers specifically in how to modify assignments and adapt materials. Observation of teacher and student performance: Six of the teachers we interviewed (20%) wanted to be observed working with students. They wanted specific suggestions about how they might change their instruction in ways that would ensure student success, although they wanted these tailored to fit their teaching style and classroom. They felt observation was a critical component in developing more effective plans, but said team members who observed, did so only for ten to twenty minutes, and usually only observed the student. In-class modeling, consultation and support: Four of the teachers we interviewed (13.3%) wanted someone to model strategies specific to the curriculum, to their instruction, and to behavior management. They did not perceive brief or out-of-class suggestions as adequate to ensure their efforts would be appropriate. They wanted personnel available to show them how to implement strategies and ensure they were . implementing them correctly. 109 Specific instruction in how to modify assignments and adapt materials: Finally, nine teachers (30%) said they did not know how to appropriately modify assignments or adapt materials to student needs. Their primary method of modifying assignments was reduction - that is, they had the student complete every other problem, or learn a portion of a spelling list, or write fewer paragraphs. They recognized that having students do less was of very little value instructionally. Teachers reported their primary method of adapting materials was to use lower level texts or to have materials read aloud. Again, they recognized this had very limited validity as an instructional method. Finally, teachers reported no real modification when they tested students. They did not believe that these modifications would help students when they were expected to perform on their unit tests — because tests would not reflect the altered content the students had received, and they would not permit these modifications during testing. In general, teachers recognized more could be done in terms of modification and adaptation of materials, but they did not know what. Although teachers we interviewed talked very explicitly about their needs, they did not see the specialists on the team as good resources for this type of professional development. They gave three reasons for this. First, teachers stated they did not think teams had the expertise to do this — because if they had, teachers thought they would have made more appropriate recommendations for interventions. Second, teachers felt the teams’ expertise was in testing and that their time would be better spent doing that “so students can get the special education help they need.” Third, some teachers said they were so dissatisfied with the teams and had so little respect for the specialists that they would not be interested in attending inservices or workshops they might hold. 110 Additionally, as much as these teachers said they wanted and needed training in these areas, and did not perceive the team members as able to provide these supports, they were also pessimistic about the district fulfilling their needs. They reported that central office personnel predetermined professional development activities in their schools with little input from teachers. The fact that teachers did not view specialists as competent to fulfill their needs and did not view the district as aware enough of their needs to address them reinforced teachers’ beliefs that the only way the students they referred would get the help they needed would be to get them placed in special education. Teachers said in order to receive the training they needed, they would have to find and enroll in classes and workshops on their own, but did not see this as likely because of competing demands on their time for other coursework or professional development activities. More Voice in Decision Making Eleven of the thirty teachers we interviewed (36.7%) said they wanted to have a stronger voice in determining whether or not a student received special education services. They did not agree with the methods used by the teams to make their determinations. They said team members relied excessively on paper-pencil tasks that did not reflect what was actually happening with the student, and did not look at test results in the context of what was happening daily with the student. Teachers repeatedly stated they wanted team members to do more observations and look at more data, and they do not see this occurring. In addition teachers do not believe they are being listened to or that their statements about what the student needs are being taken seriously. In fact, five of the III teachers we interviewed (16.7%) thought they were being deceived. Three of these five teachers said they had previously referred students who were determined to be ineligible for special education services. The students moved to other districts and were immediately tested for and placed in special education. The two other teachers who thought they were being deceived said they found themselves overwhelmed by special education jargon. One of these teachers said, In the past, when I thought that for sure the kid would make it, they didn’t. That’s why I thought I was lied to about results. I never see the tests. They won’t show you the tests, they won’t show the results, they just have it handed out written so I can’t even see. .. it just makes is a little bit harder to firdge when you have the tests in front of you. Further, while many recognized that some students’ problems were more severe than others, they thought the students they were referring were deserving and in need of support. All of the teachers said they only referred students who “truly needed help.” One teacher reported on her experiences with a particular student: The social worker said, “Well, is he wiping feces on his face, is he hurting himself, is he self-mutilating, is he making animal noises? If he’s not doing any of that stuff, then he doesn’t qualify for my services.” So we sent him back to third grade to repeat last year, as if that will help him. He can’t do the work there, either. And he is still struggling, still crying, still frustrated. The social worker was saying she doesn’t have time for me. That’s what she was saying. Another teacher, when asked if there were anything else she wanted to say about the teams, said: We don’t just refer people off the tops of our head. We refer children who need some big help and who need — the experience and the professionalism from somebody else who knows something and is perhaps more qualified than I am, and can put their finger on something. Whatever kind of testing or interaction will do that. They need to sit down more than five minutes of the day. In other words, these teachers had significant concerns about students, but did not think their concerns were being adequately considered or their needs adequately addressed. 112 They wanted the students in special education, which is understandable, given the poor quality of the plans that are being provided them. If problem identification, plan development, and procedures for implementation and evaluation were in place, and they were provided with sufficient training and support, we wonder whether their attitudes would be different. Question Four: Summary To summarize the data relative to our fourth research question: “What kinds of supports and resources do teachers say they need in order to make prereferral interventions succeed?” teachers want the teams to spend more time in the school, working with and evaluating students. They want specific skill development and service delivery options, and they want more voice in determining whether students are placed in special education. Teachers do not see prereferral intervention programs as sources for professional development and indicate that the primary responsibility of the teams should be testing children and placing them in special education. It is interesting to note that what teachers in this study describe wanting is what the literature on prereferral intervention programs says is needed if prereferral programs are to effect substantive change and positive outcomes for students. We need to know more about why teachers perceive this is not happening. It is possible, for example, the district lacks resources to implement effective problem solving programs. It is also possible district personnel are not adequately trained in problem solving and communication, or that these processes are, indeed, in place, but poor communication interferes with appropriate outcomes. II3 Most important, none (0%) of the thirty teachers we interviewed reported they received any training in team processes — collaboration, communication or problem solving. The literature says training is a critical part of the prereferral intervention process, and one that is well utilized (Whitten & Dieker, 1995). However, given that the specialists we conferred with in this district said they participated in team training, and teachers in this district said they did not, we need more information about why teachers are not included in team training. We need to know how to effectively integrate teachers into teams and provide all participants with collaboration, communication, and problem solving skills. Additional Analyses — Teacher Satisfaction Finally, although our research questions did not specifically address teachers’ satisfaction, the data we gathered from both questionnaire and interview relative to this issue provided information that is helpful in understanding teachers’ perceptions of prereferral intervention programs, and we have chosen to include it here. We addressed teacher satisfaction with prereferral intervention teams using a questionnaire item that asked teachers to indicate how satisfied they were with their experiences with the team. The results, depicted in Table 33, showed that approximately one-fifth (21.9%) were very satisfied with their experiences and approximately the same amount (20.3%) were very dissatisfied. The remainder (57.8%) were equivocal (somewhat satisfied or somewhat dissatisfied). The findings suggest teachers do not consider their experiences with the teams very satisfying. These are supported by interview data where seventy percent of the teachers (n = 21) said they were very dissatisfied with their experiences with the team, and only one (3.3%) said they were very satisfied with the team. 114 Table 33 Teacher Ratings of Satisfaction (Questionnaire and Interview Data) Frguengy Percent F rgueng' Percent Very Satisfied 28 21.9 1 3.3 Somewhat Satisfied 48 37.5 1 3.3 Somewhat Dissatisfied 26 20.3 7 23.3 Very Dissatisfied 26 20.3 21 70.0 Our data about teacher satisfaction are quite different from those found in the literature. These studies report high levels of consumer satisfaction with the procedures and outcomes of the teams (Chalfant et al., 1991; Chalfant & Pysh, 1989; Fuchs & Fuchs, 1989; Fuchs et al., I990b; Graden, Casey, & Bonstrom, 1983; Harrington & Gibson, 1986). This may well be due to the differences in the way exemplar programs and the programs in this district are implemented. Additionally, as we think about the data presented in this chapter, we can see a number of reasons it would be obvious the teachers in our study were dissatisfied: 1) their perceptions that they are not members of the team, 2) the amount of time between referral and meeting, 3) lack of time to present their cases at the meetings, 4) perceptions that they were not being listened to, 5) statements they had been given no strategies to implement or being given strategies they had already tried, 6) disagreement with whatever plan or strategies they might have been given, and 7) the perception that referrals should always result in certification and placement. 115 CHAPTER FIVE - DISCUSSION This study was undertaken in an effort to better understand teachers’ use and perceptions of prereferral intervention programs. In this chapter we summarize the context of our study, its methodology and major findings. We discuss our findings in light of the study's theoretical framework, and present our conclusions. We discuss these in terms of the implications of the results for practice and present our recommendations for further research. Summary of the Study In this section we provide the reader with an overview of what has been presented and an orientation to the remainder of the chapter. We will restate the problem and purpose of the study and review it’s theoretical framework, major questions, and procedures and data analysis. We will present the major findings of the study in terms of the statistical decisions made concerning the apparent answers to the research questions. As the numbers of difficult to teach students in general education increases, the need for intervention programs that result in positive outcomes for students is increasing. Teams of specialists working with teachers to develop plans to resolve student problems are a widespread practice. However, while teachers’ roles in these relationships are critical, we know very little about their involvement, and know even less about their perceptions of the programs and their experiences within the programs. Knowledge of teachers’ perceptions of these programs and their roles within these programs helps to more clearly define factors that can result in the development of more productive interventions. Within this context, our goals are to inform consultants, such as 116 school psychologists, of factors that need to be considered when collaborating with teachers in developing interventions. The theoretical perspectives that guided our research were those of collaboration and consultation. We know teachers express a preference for collaboration, but the majority of prereferral programs adhere to the more traditional, hierarchical models of consultation. We developed our questions and interpreted our data in the context of whether the teachers perceived the teams to be collaborative or to follow more traditional, hierarchical formats, and then, what this implied relative to their roles and responsibilities. We asked four major questions: 1) who uses the teams, 2) what are teachers’ perceptions of the problem solving processes and the communication that facilitates problem solving, 3) what are teachers’ perceptions of their own roles on these teams relative to the implementation of prereferral interventions, and 4) what kinds of supports and resources do teachers say they need in order to make prereferral interventions succeed. Although not included among our original research questions, we looked at the relationship between demographic data and teachers’ perceptions of their roles in developing interventions. We also considered data relative to teacher expressions of satisfaction. We accomplished this using two complementary methodologies - survey and interview. One hundred fifty four general education elementary school teachers from a midsize urban district completed a questionnaire about their use of and experiences with prereferral intervention programs. Teachers who had not used the teams provided information regarding why they had chosen not to use the teams. A subset of thirty 117 teachers who had used the intervention programs were interviewed to obtain more detailed information regarding their satisfaction with their roles, the processes used by the teams, and the outcomes of the interventions. The questionnaire and interview generated a variety of types of data that called for a variety of statistical tests. In general, data were addressed in three ways. First, several of the sections involved nominal data that allowed the use of fi'equencies and percentages. Second, some data were analyzed using stepwise logistic regression techniques that allowed us to predict the presence or absence of outcomes based on values of a set of predictor variables. Finally, data were analyzed using qualitative methods outlined in Bogdan and Biklen (1998). Two raters independently evaluated transcripts, using these to identify recurring themes, patterns, and critical issues within each of the three areas of scrutiny — roles, processes, and outcomes. The raters then discussed their findings and reached consensus in developing a master list of categories. Emergent theories were evaluated against the data and data scrutinized to identify alternate explanations. Briefly, our results indicate the majority of the teachers in the district use the teams. The primary difference between those who have used the teams and those who have not is that all the first year teachers in our sample were in the latter group. Teachers’ descriptions of their involvement in intervention development ranged from those who said they had passive roles and were told what to do to those who assumed positions of authority, and told others what to do. Teachers with both more experience and higher levels of education perceived themselves to be more involved in intervention development. The majority of the referrals submitted to the teams were subsequently sent 118 on for additional evaluation. Only a few teachers were actually provided with any semblance of strategies to use. Teachers want the teams to increase the amount of time they spend in the school, working with and testing students. Teachers want specific skill development and training in team policies and procedures, and want more voice in whether or not students receive special education services. While they also want specific training in working with students with academic, attention and behavior problems, they do not see the teams as resources for this type of professional development. In all, teachers’ descriptions indicate the components of the problem solving processes discussed in the literature - problem identification, plan development, and procedures for implementation and evaluation - were rarely present. In those instances where there were any of the components are present, the subsequent interventions that were developed were limited in scope and design. Not surprisingly, implementation was weak, because the strategies were so poorly designed and any information they were given about the interventions was solely procedural. Furthermore, teachers did not have the requisite conceptual knowledge to appropriately implement the interventions. However, any information they were given about the interventions was solely procedural. Since the process is seriously flawed; the product which emanates from the process is flawed. This, in turn, affects the implementation, which in turn, affects the outcomes. Satisfaction is lowered, students’ problems increase, and special education enrollments ultimately increase as well. Limitations Before discussing the questions our results raise, there are a number of factors that limit their generalizability. As stated in Chapter Three, there are the obvious limitations 119 in terms of whether the respondents in this sample are representative of teachers in other urban school districts. Further, while a recent study indicates screening teams are the most frequently employed support system in schools (Bahr, Whitten, & Dicker, 1999), we do not know whether the teams in this district are representative of teams in other districts. These sample-specific limitations necessitate replication studies. Future research is needed to extend the validity of the findings beyond the specific educational and cultural settings in which this study is imbedded. There are also methodology-specific factors that limit the generalizability of our research. Specifically, there is potential for interpretation bias when questionnaire respondents use their own interpretations of items in the absence of clear and specific definitions. There is also potential for bias because interview responses tend to be inconsistent in length and content across respondents, making it difficult to ensure standardization and quantification of results. The findings in this study may also have been influenced by methodological problems inherent in any study of “perceptions.” That is, teachers’ perceptions, albeit critical ones, represent only one set of perspectives. Perceptions are based on memory and are dependent on how one organizes information. We do not know whether others involved in the situations teachers described would agree with their descriptions, or recollect what happened in the same way. Additionally, the data were self-report in nature. Thus we do not know that what respondents report is what they do in actual practice. Observation of classrooms and of consultation interactions would be needed to ascertain whether or not respondents were answering in a manner consistent with their actual performance (Borg et al., 1993). 120 Additionally, there are limitations in the instrumentation. Our questionnaire was developed based on our review of the literature. Teachers responded to the questionnaire based on their experiences. Where terms in the questionnaire did not mesh with their experiences, teachers defined terms in their context. This was not uncovered until we began to triangulate our data, and noticed various inconsistencies. Consequently the questionnaire itself becomes a limitation of the study, as it raises questions about what the data we obtained from it mean. Future research would require redesigning the questionnaire to ensure terms were more clearly stated and universally understood. The data we obtained from our interviews should be useful in an updated version. Finally, our study was not designed to look at direct outcomes for students. We have some information about what teachers said happened, but we do not have direct information about specific students. Without direct data about specific outcomes for students that determine whether academic achievement and behavior is actually improving, and whether those gains are maintained over time, any research on prereferral intervention programs will be incomplete. Discussion and Conclusions Nevertheless, there were several interesting findings, and in this section we discuss the questions this study raises and the implications for fiiture research. In particular, we will discuss 1) what we know about the problem solving processes, and 2) what we know about teacher participation and teacher roles. 121 Problem Solving Processes In this section we discuss what we know about the problem solving processes. Specifically, we will address I) the components of the problem solving process, 2) length of the meeting, 3) intervention implementation, and 4) supports and resources. Components of the Problem Solving Processes: None of the strategies teachers described emanating from their meetings with the teams were developed using the components of the problem solving process as described in the literature. That is, problem identification was limited primarily to teacher verbal reports and a casual perusal of the students’ files. Plan development was essentially nonexistent. Typically, a team member would pronounce what would happen next afier the teacher gave a brief report of their concerns about the student. Procedures for implementation were implied rather than explicit, and there were no provisions for evaluation. We know that under carefully controlled circumstances, the components of the problem solving process can result in interventions that have positive outcomes for students. We know from the exemplar prereferral intervention programs, these can be effectively implemented in school settings (Fuchs, Fuchs, Bahr, Fernstrom, & Stecker, 1990; Graden, Casey, & Bonstrom, 19853; Kovaleski, Gickling, & Morrow, 1999; Rosenfield & Gravois, 1996). We also know that teachers are likely to implement interventions that are well developed (Kovaleski et al., 1999; Noell & Witt. 1999). While the reader might readily jump to the conclusion that these problems are specific to this district, we understand them to be endemic. Our data is like that of Flugum and Reschly (1994) and Fuchs (1987), in that it suggests that use of strategies is limited, and even when the basic components are in place, the plans that result from them 122 are of poor quality. Our field testing and pilot studies, conducted in two rural school districts and a large urban school district, indicated the only differences between teams were in who the participants were. Otherwise, teachers expressed the same concerns in pilot and field studies as they did in the study sample. We need more studies that tell us how widespread the problem with poor-quality interventions is. Further, we need to consider the circumstances under which these problems are less likely to occur. It may be that there are differences in locations where these programs are mandated rather than recommended. For example, recent research by Bahr, Whitten, and Dieker (1999), says that these programs are higher quality in Illinois than in Indiana or Michigan. Earlier studies indicated these programs were mandated in Illinois, but not in Michigan (Bahr, 1994; Carter & Sugai, 1989). It is our understanding that the status of those mandates remains the same at this time. However, the Bahr study consisted of survey research that did not explicitly include referring teachers as respondents. Nevertheless, this work may serve as a foundation for looking at how different programs may have different impacts on teacher perceptions. Length of the Meeting: As mentioned repeatedly in this paper, team meetings lasted only ten minutes. We do not know why they are so limited, as the time allotted is not sufficient to engage in the problem solving process. However, providing more time is not necessarily the answer either. Time is a necessary but insufficient condition for realizing quality. There is precedent in the literature to use brief meetings to screen problems for prereferral in order to provide for gathering of additional information (Conoley & Conoley, 1982). However, those programs advocate screening as a first step, rather than the sum total of the problem solving process. We need research on how to use 123 available time efficiently and effectively. This suggests research that observes how personnel in effective programs utilize and manage time. Intervention Implementation: In the first chapter, we said the Zeitgeist of intervention implementation research suggested teachers stand in the way of appropriate implementation. Several of our findings related to this. First, the teachers in our study were almost never given strategies to implement. Second, the strategies or plans teachers in our sample were given were so poorly designed as to be worthless. Third, while our sample was very small, three of the four teachers attempted to implement the interventions they were given, even when they disagreed with them. Further, their comments suggested they did not have the conceptual knowledge to implement the interventions, and had received only procedural information. Intervention implementation is the core of any successfiil prereferral intervention program. We need to know a lot more about how to ensure effective intervention implementation that results in direct and positive outcomes for students. We need to know a lot more about the interventions themselves. We need to know what works and what does not work, given the logistics of particular interventions. We need to know more about the constraints teachers operate under, and how their other responsibilities impact on their being able to realistically implement certain types of interventions. Intervention implementation is where we must focus if we are going to get the outcomes we say we want. However, we cannot have implementation without plan development, and we cannot have plan development without problem identification. The question does not appear to be how to get teachers to implement interventions, but rather, how to ensure they get quality interventions to implement. 124 We don’t need to know more about the problem solving process merely as a set of procedures. Developing programs that result in appropriate intervention implementation is more complicated than just following steps or talking to one another. We do need to know more about the process of developing, implementing, and evaluating interventions. We need a lot more research on determining what teachers need to implement interventions and what happens to the students as a result of providing teachers with these supports. This requires knowledge of curriculum, strategies, and the student. Supports and Resources: As the few interventions teachers were given to implement were inappropriate, it is pointless to discuss the supports and resources teachers might have needed to implement them. It is interesting to note that teachers repeatedly expressed interest in having training and service delivery options like those described in the literature on prereferral intervention. That is, they said they wanted to be provided with interventions that effectively resolve student problems, they wanted people in their classrooms, modeling new techniques, and observing and critiquing them implementing the interventions. If teams are to provide effective prereferral systems, they must provide teachers with adequate supports and resources for implementing the interventions. Nevertheless, it is critical to keep in mind that adequate supports and resources require a meaningful context. There is no point in providing teachers with support unless the interventions are meaningful. However, before we can say with any authority what teachers need in terms of support and resources, we have to know more about implementing interventions that result in direct positive outcomes for students, and what teachers need to implement interventions that work. 125 Teachers’ Roles and Participation In this section we discuss what we know about teachers’ roles and their participation with the teams. Specifically, we will address 1) first year teachers, 2) team membership, 3) professional development, and 4) team training. First Year Teachers: One of our findings is that first year teachers in our sample did not use the teams. This is a concern, given that this is a time at which these teachers are most likely to need help, one would think prereferral intervention programs could be a tremendously powerful support system. The literature provides virtually no information about characteristics of consultees, and we do not know whether these patterns of use are typical in other districts or with other types of prereferral intervention programs. We do not know why the first year teachers in our sample did not use the teams. As we do not know whether these ten teachers are representative of other first year teachers in the district, of other first year teachers in general, our findings are subject to replication. If our findings hold true, we need more information about why first year teachers do not use these teams. We need to know how new teachers learn about these services and how they might best utilize them. Specifically, we need research to determine whether our findings hold true across districts, across teams, and across programs. We might ask first year teachers whether they know about the teams, whether they know under what circumstances they can refer students and what happens when they make a referral. If, for example, first year teachers do not know about the teams, we need to find better ways to ensure they learn about them. If research indicates first year teachers know about the teams but do not use them because they have not been able to sort through an overwhelming set of responsibilities to 126 do so, we need to ensure they receive better mentoring and support. These might include small group sessions or individual contacts initiated by team members. These might be integrated more specifically into the district’s teacher orientation, or mentoring programs might be more formalized and include informal discussions with team members, when the first year teacher is not in the midst of an actual referral. More generally, research might examine how and when teachers learn about these teams, and about various school support systems, how they translate the information they obtain into practice, and how they perceive they might have been better oriented to the processes, both covert and explicit. It should not be surprising that we found a relationship between teachers’ perceptions of their involvement in developing interventions, and in their levels of education and experience. However, we need to know more about how we can accelerate, encourage, and enhance involvement. We do not know, for example, whether general education teacher candidates are exposed to these teams during undergraduate training, particularly their practica or internships. The fact that many teachers conclude if they knew the appropriate jargon their student would be tested seems to be symptomatic of the problems in communication with program participants, and needs more study. If for example, teachers do not understand the referral processes, and are being told some referrals are appropriate and some are not, there is a basic problem. The terms “accurate” or “appropriate” referrals suggest teams are not adhering to the tenets of prereferral intervention that say they are designed to provide support for any teachers who need help for any student. Team Membership: None of the thirty teachers we interviewed considered him or herself to be a member of the team, although virtually all the teachers attended the 127 meetings and presented information at them. When we discussed team membership in Chapter Two, we cited literature that said the referring teacher is a member of all teams in some capacity, but that their roles had not been clearly defined. While again, we do not know the extent to which programs in this district are representative of programs in other districts, it appears at best, the capacity in which our teachers are involved in the programs is very limited. As in the literature, teachers in our sample described a hierarchical program in which the specialists were experts to whom they went for an expert decree. This lack of teachers’ perceptions of themselves as members of the team is disappointing because it is diametrically opposed to the literature that emphasizes the need for including the teacher as an active and empowered member of the process (Adelman & Taylor, 1998; Graden, Casey, & Christenson, I985b; Knoff, Hines, & Kromrey, 1995a; Knoff, Sullivan, & Liu, 1995b). If this is the case generally, we need more research about team membership and roles. If teachers do not perceive themselves as central members of the teams, it is unlikely that they could contribute fiilly and invest in the decisions that are made. We need to know whether this is specific to our sample. We need to know why teams perceive specialists and teachers as a “we-they” phenomenon. We need to know what changes in team structure would lead to positive changes, and whether collaborative structures are more likely to lead teachers to consider themselves active and empowered team members. Professional Development: One of the major goals of prereferral intervention programs is that of professional development. That none of the teachers we interviewed were provided with information or strategies they found helpful would suggest this goal 128 is not being accomplished. While providing professional development is a stated goal of prereferral intervention programs in virtually every work we found on the practices, we found very little information that relates to these teams as venues for professional development. We see tremendous potential for prereferral intervention teams being involved in professional development, but need research to determine how this is best accomplished. We need research in whether different models of prereferral intervention are more likely to enhance skills, or whether different models are more likely to result in generalizing skills. We need research in how to effect professional development through these teams. For example, everyone involved needs to know there is a specific problem solving process to follow, and everyone involved needs to be trained in these processes. However, as they go through the processes, they must have appropriate content. The typical interventions teachers use and psychologists recommend are generic and limited in scope. There are programs available that provide more detailed, specific and varied content, such as Project RIDE (Beck, 1991). Research might examine the extent to which using these prepared strategies leads to better implementation. Professional development involves more than just learning new strategies and problem solving skills. It involves conceptual and practical preparation that makes schools better equipped to teach students successfully and allows professionals to work together for the benefit of the student. Teams should be a source of knowledge that becomes formalized, codified, and shared within and across buildings. They should be a source for providing a knowledge base that results in competent teacher leaders who problem solve and collaborate with other teachers and other specialists. 129 Team Training: Not only do we need to examine the role of teachers as team members; we need to develop a better understanding of why none of the thirty teachers we interviewed said they had training in prereferral intervention processes. In a survey of these teams, Whitten and Dieker (1995) report high levels of participation in team training. However, we do not know what that training involved, particularly as the majority of respondents in that study reported they needed additional training in problem solving, communication, and collaboration. Further, we do not know whether the referring teacher was included in that training. All the teachers we interviewed were very specific that the only training they had was a when team members introduced themselves at a staff meeting, and told them how to find and fill out the referral form with their name and the students’ name and type of problem. At the same time, the specialists we spoke with described having participated in an elaborate training program that emphasized problem solving, collaboration, and communication, provided by the district. We need to know more about what training specialists and referring teachers do receive. We need to know more about what sense they make of this training and how and in what ways the training impacts on their ability to design and implement interventions that have a positive affect on direct student outcomes. It appears that, for whatever reasons, administrators do not view referring teachers as part of the team. It is understandable that districts would invest their resources in specialist training. At the same time, that underscores the inequity in the teachers’ role on these teams. We need to know more about how to ensure teachers are included in the processes. We think it is more complicated than just following the recommendations of 130 the literature that specialists ask teachers for their opinions or input, or that specialists not make judgmental statements about the teachers’ choices and decisions. We need to understand how the teachers’ expertise fits in the context of optimal team fiinctioning. We need to know more about how referring teachers who do participate in team training (for example, as part of the Peer Collaboration program) perceive their experiences with teams. We need to know whether training allows teachers to be more a part of the process, and if so, whether it increases their participation and results in successfiil implementation of interventions that have direct and positive outcomes for students. Characteristics of Referrals - Gender, Retention, and Referral Data Although we did not discuss it in our results section, the data presented in Appendix G regarding characteristics of student referrals yielded some noteworthy results. Summarized here briefly, data from the thirty teachers we interviewed indicate teachers refer approximately ten percent of the students in their buildings each year. Of the thirty students described by the teachers we interviewed, 63% were male, 37% had previously been retained, and 41% had been referred to the teams before. A common concern in the literature on prereferral intervention is that we do not know whether the students being served by P1 teams are substantially different from those who are referred to special education. Our results suggest these may be the same students, at least in teachers’ perceptions, as only two teachers we interviewed said they made referrals to the team for concerns other than special education eligibility. While there is a lot of data in the literature about the types of problems for which teams design interventions, there is very little information available about specific characteristics of student referrals. We were able to find no information about referrals to prereferral 131 intervention programs by gender, retention, or referral history. It seems reasonable to think that if we knew more about the specific students being referred, we might be able to do a better job of targeting prereferral and professional development activities. Gender: We were able to obtain some current demographic information from the State Department of Education that said that males composed 67.8% of the students certified in special education in the categories that would most likely be participating in general education classes (EI, EMI, LD, POI-II, or SLI). This is interesting in its similarity to our findings, where teachers said 63% of the students they had referred most recently were male. However, we continue to need research about who is referred to prereferral intervention teams, and whether or how they differ from students who are referred to —— or placed in - special education programs. Retention: Teachers we interviewed reported that 37% of the students they referred to the teams had already been retained in grade at least once. While we had thought it common knowledge that a large portion of special education students were retained before they were placed in special education, we were not able to find any data that supports this. If the retention rates are this high for students sent to prereferral, and high as well for students in special education, then we need to examine the relationship between retention, referral to prereferral, and special education certification more systematically. We need to study why this relationship exists, whether we can predict what students will not succeed with retention, and more important, whether psychoeducational evaluations should be prerequisite to retention. Recycling: According to reports of teachers we interviewed, 41% of the students they referred had previously been referred to teams. It is common folklore that students 132 are “recycled” through these prereferral intervention teams - that is, teachers refer them to the teams, are sent away with interventions to implement, and return to the team six months to a year later, reporting no changes in student performance. Our data would suggest this is more than just a myth. However, we found nothing in the literature that addresses this type of cycling. We need research to validate our results. More important, we need to know why so many students are being referred repeatedly, and what this “cycling” has to do with the structure and functioning of the team and the nature of the problems themselves. In short, extensive additional research is needed in terms of characteristics of students referred to prereferral intervention programs. Closing Comments: Reflections on the Current Study Finally, complementary methodology used in this study led to fortuitous findings that have implications for future research. When the results we obtained on the interviews appeared to contradict what we found on our questionnaires, our search to discover why led us to uncover the fact that teachers were interpreting questionnaire items differently than the way in which we had intended. During the interviews, the real intent of the questions became clarified as we conversed with teachers. For example, we learned that while teachers did want the specific student they referred to be in special education, on a general basis, they wanted to be better teachers, ones who could teach students in their classrooms more successfully. Had we not gathered data from more than one source, we would not have realized the extent to which subtle semantic differences might influence teachers’ responses, as well as our interpretation of our results. Rather, those inconsistencies provided us with a much broader vision of teachers’ perceptions, and allowed this novice researcher to understand, in greater detail, the strengths and 133 limitations of both types of research. For example, the results of information obtained in the pilot and field studies could have been used much more effectively. Rather than simply providing us with information about whether the questions were understandable and clear, or how long it took to administer the instruments, more thoughtful interaction with the data we obtained through those field studies would have helped improve the research design. And, while we based our interview design on the literature and what we expected to learn from the questionnaire responses, it would have been equally beneficial if we had used interviews to design the questionnaires. The contexts in which our study was planned and conducted, and the nuances of language that resulted in problems with our collection of data to corroborate one another is a problem that needs to be smoothed out in future research. Using different methodologies such as scenario research, direct observation, and longitudinal studies, in which specific students are followed from teacher concern to problem resolution, or specific teams are followed across the course of the year, with input from all participants would be useful. If specialists such as school psychologists are committed to changing their role from that of “testing” to that of providing effective supports and services for students, as the National Association of School Psychologists claims, they need to work much more actively at changing their performance and their image. Providing proactive support to new teachers would be a wonderful way to do this. If prereferral teams were to be proactive, it would seem that specialists might seek out and support those newest teachers. More emphasis on the “professional development” component of the stated goals for prereferral intervention is needed. We suspect this is not the result of a single problem, but of multiple factors that can only be resolved through changes in training 134 programs and professional development for those who implement these teams. We need to ensure that all participants, including the referring teacher, are more than just familiar with the processes, and know more than just procedural information. We need to ensure they are using these processes correctly and consistently. This requires ongoing professional development for all participants. It requires better supervision and monitoring, and requires that participants be held accountable in their involvement. Prereferral intervention is a program that has enormous potential. Realizing this potential will require tremendous effort, dedication, and knowledge. Hopefully this dissertation is one step toward that end. 135 APPENDICES 136 APPENDIX A QUESTIONNAIRE 137 PREREFERRAL INTERVENTION PROQRAMS - SURVEY INFORMATION AND CONSENT This survey has been designed to help us learn about your experiences — positive or negative - with prereferral intervention programs in your school. Prereferral Intervention/ Student Support Teams utilize small groups of professionals who attempt to solve problems by developing interventions which best meet the needs of children in school. Solving problems before they become severe enough to require special education (pre-referral intervention) is the major goal of these teams. We need to determine to what extent and in what ways, these programs are effective. (Zour district ragers to these as: ) The survey should take THREE TO FIVE minutes to complete. Your participation is voluntary, you may refuse to participate or answer questions, or discontinue your participation at any time without penalty. Your responses will not be identified and confidentiality will be maintained in any report on these findings. You indicate your voluntary agreement to participate By completing and returning this questionnaire. THANK YOU Questions or concerns should be directed to Amy Pobst 138 PLEASE RESPOND TO THE FOLLOWING: 1) Grade level currently teaching: 2) Number of years teaching that grade: 3) Total years of teaching: 4) Identify your current teaching assignment: Special Ed._ General Ed. _ Other 5) Circle the highest degree you have obtained (indicate additional credits if appropriate): BA BA+ MA MA+ Ed.S. Ph.D. 6) Are you also certified in special education? No [ ] Yes [ ] Endorsement? EI_ LD_ MI_ Other— PLEASE RESPOND TO THE FOLLOWING QUESTIONS ABOUT THE PREREFERRAL INTERVENTION TEAM IN YOUR SCHOOL 7) Have you ever used the prereferral intervention/child study team in your school? No [ ] Yes [ ] (If your answer is yes, please go on to the next page). 8) If you have never used the prereferral intervention/child study team in your school, why not? I am a first year teacher I have never needed support services I used other support services (which ones? I understand from other teachers that the team is not helpful. Other (If you have never used the team, please stop here and turn in your survey. Thank you). 139 9) During this school yeg ( 1999-2000) AND Last 0998-99) school year, how many of your students were referred to this team (i.e. referred by you, by the students’ parents or guardians, or by another teacher who works with the student)? None One Two Three More than three 10) If you referred no students during this or the last school year, when was the last time you used the team? l997-98 1996-97 I995-96 Prior to 1995 11) If you were not involved in the referrals of any students to this team within the last year, why not? (Check all that apply) I did not need support services I used other support services (which ones? ) __ I have used the team in the past and did not find it helpful. __ Other THINK ABOUT THE LAST STUDENT YOU DISCUSSED WITH THE TEAM AS YOU ANSWER THE FOLLOWING UESTIONS 12) Was the student’s problem mainly: Academic _ Behavioral _ Both _ Other 13) Who attended the meeting? (Check all that attended) _ I did _ School Psychologist __ Other General Ed. Teachers _ School Social Worker _ Special Ed. Teacher/Consultant __ Principal/Asst. Principal Parents __ Others: 14) Check the person most responsible for developing interventions. (Check only one) __ I was _ School Psychologist Other General Ed. Teachers School Social Worker Special Ed. Teacher/Consultant Principal/Asst. Principal Parents Other: 140 15) 16) 17) 13) 19) What was your role in the development of the interventions? I had no role in the development of the interventions. I presented information about the student and was asked questions, then I was told what interventions I should implement. The other members developed interventions. I was asked to select the best ones. I collaborated with colleagues in the actual development of the interventions. I assumed a leadership role in developing interventions. Other To what extent did you agree with the plan that was developed? Agreed _ Agreed _ Disagreed _ Disagreed Completely Partially Partially Completely Check the person most responsible for implementing the interventions? (Check only one) I was _ School Psychologist Other General Ed. Teachers _ School Social Worker Special Ed. Teacher/Consultant _ Principal/Asst. Principal Parents _ Other: If you were primarily responsible for implementing the interventions, did you? Yes [ ] No [ ] Istarted to, but stopped [ ] If no, why not, OR if you started to, but stopped, why? (Check only one) They took too much of my time. They were not appropriate. They were not fair to the other students. I did not have the background or training to implement them. They made the problems worse. Another strategy was more successful. Other: 141 20) 21) 22) 23) 24) To what extent was the student’s problem resolved? Fully _ Mostly __ Partially _ Not Resolved or Resolved Resolved Resolved became Worse If the student’s problem was not fully resolved, what happened? (Check only one) The student moved on to the next grade The student continued to have problems The student was referred to special ed., but it was too late in the year for testing. The student was tested for special education, but did not qualify The student was tested for special education, and was placed in special education Other How satisfied were you with your experience with the team? (Check one) Very Somewhat _ Somewhat __ Very Satisfied Satisfied Dissatisfied Dissatisfied How likely are you to refer more students to this team? (Check one) Very __ Somewhat _ Somewhat __ Very Likely Likely Unlikely Unlikely How likely are you to recommend the team to other teachers? (Check one) Very __ Somewhat _ Somewhat __ Very Likely Likely Unlikely Unlikely 142 Your views are very important to us. We would also like to interview some teachers about their experiences - both positive and negative — with Prereferral intervention programs, to get more information about how these might work best. Are you willing to be interviewed; Yes [ ] Possibly, I’d like to know more first [ ] No [ ] 0 Interviews will take place during December and January, and will take about half an hour — time scheduled at your convenience. 0 If you are willing to consider being interviewed (you may change your mind at any time!), please complete the following: I can be reached at (phone) or (e-mail) The best times to reach me are: My name is: (Please print) 6 Thank you again for your help! 143 APPENDIX B INTERVIEW FOR FORMER USERS 144 Prereferral Intervention Interview — Former Users Code: Date: MAKE SURE: Go back through the survey and confirm/clarify the background information, years teaching, endorsements and experience, etc. Reminder of confidentiality, ability to stop proceedings and/or tape at any time. Signature on permission slip. You said on your survey that you did not use the prereferral intervention team in your school, giving as your reasoning that: You did not need support services You used other support services (which ones? ) You had used the team in the past and did not find it helpful. Other 1 I) Did not need support services I 2) Can you tell me why you didn’t need them? (i.e. students well behaved, was able to solve any problems that arose on my own, worked them out with ) 3) Did you find the team helpful when you used it? 4) Will you use the team again? 5) What do you think is needed for those teams to be more effective? 6) What (time, personnel training, materials, etc.) do teachers need to be able to successfully implement the interventions recommended? 7) Is there anything else you think I should know about your experience with the team or prereferral process in general? [ 8) Used other support services (which ones? ) 1 9) Can you tell me the reasons why you used (this other service) rather than the prereferral intervention team? I45 10) Will you use the team again, or continue to use this other resource? 11)Did you find the team helpful when you used it? 12) What do you think is needed for those teams to be more effective? 13) What (time, personnel training, materials, etc.) do teachers need to be able to successfiilly implement the interventions recommended? 14) Is there anything else you think I should know about your experience with the team or prereferral process in general? 15) You had used the team in the past and did not find it helpful. 16) Can you tell me what happened? 17) Do you think you will use the team again? 18) What do you think is needed for those teams to be more effective? 19) What (time, personnel training, materials, etc.) do teachers need to be able to successfiilly implement the interventions recommended? 20) Is there anything else you think I should know about your experience with the team or prereferral process in general? 21)Used the team in the past and did not find it helpfiil. 22) Can you tell me what happened that it wasn’t helpful? 23) What do you think might have improved your experience? 24) What do you think is needed for those teams to be more effective? 25)What (time, personnel training, materials, etc.) do teachers need to be able to successfully implement the interventions recommended? 26) Is there anything else you think I should know about your experience with the team or prereferral process in general? Does your school/team have training about how to use the teams? What kind of support is available to teachers in implementing the interventions? Are there any provisions made for you to get help if things aren’t working right? 146 APPENDIX C INTERVIEW FOR RECENT USERS 147 PREREFERRAL INTERVENTION PROCESSES INTERVIEW - INFORMATION This interview has been designed to help us learn about your experiences — positive or negative - with prereferral intervention programs in your school. Prereferral/Intervention/Child Study teams utilize small groups of professionals who attempt to solve problems by developing interventions which best meet the needs of children in school. Solving problems before they become severe enough to require special education (pre-referral intervention) is the major goal of these teams. We need to determine to what extent and in what ways, these programs are effective. IN FQRMED CONSENT FOR INTERVIEW We wish to make clear that you have the right to refuse to be interviewed. Further, you may discontinue your participation at any time. You may also withdraw your permission for us to use any information gathered for research purposes. Also, we want you to know that your participation (or non-participation) will NOT in any way be used to penalize you. All data we collect will be stored in a secure place, accessible only to project staff. In writing research reports produced from this work, we will take every precaution to ensure that the information presented cannot be linked to any specific participant(s). I am willing to be interviewed. I am not willing to be interviewed. (Name/ Signature) (Date) 148 PREREFERRAL INTERVENTION INTERVIEW — RECENT USERS Code: Date: MAKE SURE: Go back through the survey, confirm background information, years teaching, endorsements and experience, etc. Reminder of confidentiality, ability to stop proceedings and/or tape at any time. Signature on permission slip. 10. ll. . Before we start, I’d like to ask you how many students you typically refer to the team in any given year? For this interview, I want you to think about the last student you referred to the team. Was this student male [ ] or female [ ] ? Was s/he typical of the students you’ve referred to the team? Yes [ ] No [ ] (If the student is NOT typical, what is a more typical student like? Use the typical student for the remainder of the interview) Could you give me his/her first name? What grade was s/he in? Had s/he ever been retained? Yes No DK Was this the first time s/he was referred? Yes No DK Why did you refer him/her? Academic Behavioral Both Could you describe briefly the events or issues led you to refer him/her? (Can you describe the concerns you had about his/her academic/behavior problem that led you to refer him/her?) Before you referred him/her, what types of things had you done to try to solve the problem? NOW, I’D LIKE YOU TO THINK ABOUT THE TIME AFTER YOU REFERRED THE STUDENT, BUT BEFORE THE MEETING. Did anyone collect information? Who did that? (Observation? Interview — student, teacher, parent? Pre—testing? CBA, FBA) 12. NEXT, I’D LIKE YOU THINK ABOUT THE MEETING ITSELF. WERE YOU THERE? YES [ ] NO I ] (Ifno, why not?) IF YES, I’D LIKE YOU TO TELL ME ABOUT THE MEETING. 149 l3. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23. 24. Could you tell me what happened first? (Problem identification, analysis, clarification). Then what happened? (Plan development, procedures for evaluation?) How did the meeting end? (Were all factors in place — who was responsible for what, when, what outcomes would be assessed?) During the meeting, what kind of information did you talk about or look at? (verbal, file review, work samples, discipline referrals, medical information? — Any “hard” data — baseline information?) What did the team recommend be done? (Very specifically, what was the plan?) How did the group decide what intervention(s) would be recommended? What did the planning “look” like? (Did you talk about different interventions and whether they would be possible?) What was your role in deciding what interventions would be used? Was there a written — a formal — plan? If there was no written plan, did anyone take notes/record what was going to happen? Who was supposed to implement the plan? If implemented by another, was the plan implemented? Why or why not? If you were the implementer, were you able to implement the plan? Why or why not? Did you know how to do what was asked of you? What kind of support was available relative to implementing the intervention? Was there any training or support available to you? Were there any provisions made for you to get help if things weren’t working right? Were there provisions made for follow-up to determine whether the plan was effective? (Baseline data? Another meeting scheduled? Contact by team members? Contingency plans?) If you were able to implement the plan, please tell me what you did (get lots of detail). 25. In your opinion, did the plan work? Yes [ ] No [ ] To what extent? Why or why not? 150 YOU’VE TOLD ME A LOT ABOUT WHAT ACTUALLY HAPPENED. NOW I’D LIKE YOU TO TELL ME WHAT YOU THOUGHT ABOUT IT ALL. 26. What did you think about the plan? Did you think it would work? Did you think it was appropriate? Why? Why not? Had you already tried those interventions? If so, did you tell them? How did they respond? 27. What did you think about HOW the interventions were developed? Were you adequately involved? Were there options available to you if you disagreed with what was recommended? 28. What happened to/with the student? Were his/her problems resolved as a result of this intervention? Did this prevent his/her referral to special education? 29. Were you satisfied with your experience? What did you think should have happened? Were you treated professionally? 30. What kinds of resources (time, personnel training, materials, etc.) do teachers need to be able to successfully implement the interventions recommended? 31. Did you receive any training on the functions and use of prereferral intervention teams? What did that consist of? 32. “Model” prereferral programs are described as involving/treating/including the referring teacher on a continuum, from the teacher as the recipient of the expert knowledge of the other team members at one end, to teachers being asked questions and told what to do, to teachers being asked questions and being given a few choices of options from which to choose, to teachers being the catalyst for change, with the other team members only providing support and structure in which the teacher solves the problem. Where do your experiences fit? 33. Is there anything else you think I should know about your experience with the team or prereferral process in general? 151 APPENDIX D INFORMATION AND CONSENT FOR QUESTIONNAIRE 152 PREREFERRAL INTERVENTION PROGRAMS - SURVEY INFORMATION AND CONSENT This survey has been designed to help us learn about your experiences — positive or negative - with prereferral intervention programs in your school. Prereferral Intervention/Student Support Teams utilize small groups of professionals who attempt to solve problems by developing interventions which best meet the needs of children in school. Solving problems before they become severe enough to require special education (pre-referral intervention) is the major goal of these teams. We need to determine to what extent and in what ways, these programs are effective. (Zour district regers to these as: 2 The survey should take THREE TO FIVE minutes to complete. Your participation is voluntary, you may refuse to participate or answer questions, or discontinue your participation at any time without penalty. Your responses will not be identified and confidentiality will be maintained in any report on these findings. You indicate your voluntary agreement to participate By completing and returning this questionnaire. THANK YOU Questions or concerns should be directed to Amy Pobst 153 APPENDIX E INFORMED CONSENT FOR INTERVIEW 154 PREREFERRAL INTERVENTION PROCESSES INTERVIEW - INFORMATION This interview has been designed to help us learn about your experiences — positive or negative - with prereferral intervention programs in your school. Prereferral/Intervention/Child Study teams utilize small groups of professionals who attempt to solve problems by developing interventions which best meet the needs of children in school. Solving problems before they become severe enough to require special education (pre-referral intervention) is the major goal of these teams. We need to determine to what extent and in what ways, these programs are effective. INFORMED CONSENT FOR INTERVIEW We wish to make clear that you have the right to refuse to be interviewed. Further, you may discontinue your participation at any time. You may also withdraw your permission for us to use any information gathered for research purposes. Also, we want you to know that your participation (or non-participation) will NOT in any way be used to penalize you. All data we collect will be stored in a secure place, accessible only to project staff. In writing research reports produced fiom this work, we will take every precaution to ensure that the information presented cannot be linked to any specific participant(s). I am willing to be interviewed. I am not willing to be interviewed. (Name/ Signature) (Date) 155 APPENDIX F INDICATION OF WILLINGNESS TO CONSIDER PARTICIPATING IN INTERVIEW 156 Your views are very important to us. We would also like to interview some teachers about their experiences - both positive and negative — with Prereferral intervention programs, to get more information about how these might work best. Are you willingto be interviewed? Yes [ ] Possibly, I’d like to know more first [ ] No [ I 0 Interviews will take place during December and January, and will take about half an hour — time scheduled at your convenience. 0 If you are willing to consider being interviewed (you may change your mind at any time!), please complete the following: I can be reached at (phone) or (e-mail) The best times to reach me are: My name is: (Please print) 0 Thank you again for your help! 157 APPENDIX G CHARACTERISTICS OF STUDENT REFERRALS 158 Characteristics of Referrals Although this study was not designed to address questions outside of those related to teacher use and perceptions of prereferral intervention programs, we collected a series of data that addressed the question, “What are the characteristics of teacher referrals?” Specifically, we gathered data from both the questionnaire and the interview related to: l) the numbers of students referred to the teams, 2) the types of problems students exhibit, 3) student gender, retention and referral histories, and 4) what teachers describe having done to resolve problems before referring students. These data are presented in this Appendix. Numbers of Students Referred On the questionnaire, teachers were asked to indicate the number of students with whom they worked who had been referred to the team during the previous eighteen months. Table G-l contains these data. Half the teachers (50%) referred one or two students, the other half (49.3%), referred three or four students. During interviews, teachers said they typically referred two to three students any given year, with as many as six students referred in particularly challenging years. Table G-l Numer of Students Referred to Teams - Current and Previous School Year (Questionnaire Data) Number of Students Fregueng' Percent l 31 24.2 2 33 25.8 3 29 22.7 More than 3 34 26.6 159 Types of Problems Referred The literature reports a wide variety of types of referrals to prereferral intervention programs that include poor work completion, low general achievement, specific skill deficits, failure to follow directions, language problems, distractibility, disruptive behavior and poor motivation (C halfant & Pysh, 1989; Johnson & Pugach, 1991; Korinek & McLaughlin, 1996). Data in this section are complicated to report and difficult to understand. This is, in part, due to the way problems are categorized in different studies. We asked teachers whether the type of problem exhibited by the student they most recently referred was academic, behavioral, or both, on both our questionnaire and interview. Table G-2 contains these results. Table G-2 Types of Problems Referred Qpestionnaires (n=128) I_n_terviews (n=30) Type of Problem Frgueng' Percent Frguengy Percent Academic 52 40.6 8 26.7 Behavior 36 28.1 8 26.7 Both 39 30.5 14 46.7 Missing 1 0.8 - -- Gender, Retention, and Referral Histories We were unable to find information in the literature specific to individual student referrals to prereferral intervention teams. Therefore we asked teachers who were interviewed to provide us with gender, retention history and referral histories of the students they most recently referred to the team. These data are presented in Table G-3. 160 Table G-3 Gender. Retention and Referrafilistory of Students Referred (Interview Dat_a) Frgueng' Percent Gender of studerp referred Male 19 63.3 Female 11 36.7 Studeat Previous_ly Retained in Gra<_I_e'_.7 Yes 11 36.7 No 18 60.0 Not Known l 3.3 Studen; Previoualy Referred to Tea_m_? Yes 12 40.0 No 18 60.0 Prior to Referral We asked the teachers we interviewed to describe what they had done to try to resolve problems before referring the student to the team. Table G-4 provides data related to the number of interventions attempted. The total referrals are broken down according to whether the presenting problem was 1) academic, 2) behavioral, 3) a combination of academic and behavioral problems. All the teachers (100%) reported having implemented at least two interventions prior to making a referral. A closer look at the data reveals that 30% of the teachers reported trying two to three strategies, 37% reported trying four or five strategies, and 33% reported trying six or more strategies prior to making a referral. When analyzing the interventions tried by type of referral, teachers reported using an average of 3.75 interventions when describing students with academic problems, four interventions for behavior problems, and 5.5 interventions for combined academic/behavior problems. 161 Table G-4 Number of In_terven_tions Teachers Reported Implementing Prior to Referring Studeat (Interview Da_ta) Me of Presepting Problem Total Academic Behavior Combined Academic and Behavior Numpcr of Strategies Attempted Frequency Percent Frgueng' Percent F uen Percent F uen ' Percent 2 5 16.7 -- -- 2 28.6 I 7.1 3 4 13.3 1 16.7 - -- 2 14.3 4 6 20.0 2 33.4 3 42.8 1 7.1 5 5 16.7 I 16.7 - -- 4 28.6 6 3 10.0 I 16.7 1 14.3 I 7.1 7 2 6.7 -- - - -- 2 14.3 8 3 10.0 I 16.7 - -- 2 14.3 9 l 3.3 -- -- -- -- 1 7.1 10 l 3.3 -- -- I 14.3 -- -- TOTAL 146 30 32 77 Interventions for academic problems included activities such as grouping students, providing additional help, adjusting academic assignments, requirements, and instructional content, affective strategies such as praise and encouragement, and returning a student to a lower grade. Teachers used strategies that were typically nonexclusive and required little planning and preparation relative to the individual student. Interventions for behavior problems included grouping the students with others (usually to provide closer adult supervision), removal and/or some form of punishment, parent and principal contacts, and a variety of non-specific behavior plans, rewards, and support strategies. Table G-S lists the interventions teachers said they used with students. Those who described students with both behavior and academic problems used combinations of strategies from both lists. 162 Table G-5 Staategies Teachers Used to Addres_s Problems before Refem‘arg Students to Prereferral Intervention Teams I_ntervenltions for Aczfiemic Problems Grouping — 19 Small group — 4 Cooperative learning - 2 Large group — 2 Partner with other students — 6 Pair reading - 1 Buddy to help — 1 Surround with best workers — 2 Lower frustration math group - l Additioaal Help - 35 More frequent checking with teacher — 1 One to one with teacher - 8 Title 1 teacher — 9 Work with paraprofessional. student teacher. Practicum students. volunteers — 16 Tutor at home (provided by parent) - l Affective Strategies — 6 Take her home with me — 1 C oaxing - 2 Praise — 1 Try to get him to be more outgoing — 1 Take an emotional interest in him - 1 Adjusted Requiremants and Assigpments - 11 Lower level spelling words — l Five-word club. ten word club (spelling) — l Shortened or modified assignments — 2 Oral quizzes — 2 Wait time before expecting an answer - 2 Dictating work — 2 Oral reading - 1 Adjusted Ins_tructional Content — 8 Paper organizers (structures) — 2 Use a variety of curricula - 2 Centers for reading and math - l Leveled reading books — non-frustration level — 1 Examples of others’ work to follow - 1 Extra and extensive review - l Retention - 1 Sent back to previous grade — 1 I_atervenjions for Behavior Problems Support Strategies — 7 Calling him at home every night — 1 Talk to the student - 2 Drive him home -— 1 Eat lunch with student — 1 Worked on self-esteem - 2 Contact with Parents, Princiml - 12 Notes home — 2 Phone calls home — 3 Behavior checklists sent home - 1 Meetings with parent(s) - 4 Send to principal - 3 Mints—43 Smiley faces -— 1 Stickers — 2 Reward - 2 Positive reinforcement - 2 Praise — 1 Removal/Puflshmenj — 19 Move away from other students - 3 Time out - 2 Missing all or part of recess - 1 Missing field trips. special events — 3 Escort in hallways - 1 Detention — 2 Seat next to teacher - 7 163 T_able G-5 (continued) Behavior Plans - 9 Work with others - 10 Charts on the desk - 2 School Counselor — 5 Ignoring misbehavior - 1 One on one with teacher - 5 Trying to anticipate problems and distract him — 2 Points — 4 Summary In this section we summarize what we learned about the characteristics of the students teachers refer to prereferral intervention teams. Approximately ten percent of students in schools are referred to the teams in a given year. Almost two-thirds of these referrals (63%) are male. Over one third (3 7%) have already been retained in a grade placement, and over two-fifths (41%) have previously been referred to the teams. Students are referred for a variety of academic and behavior problems, many with combined academic and behavior problems. Teachers report attempting a variety of interventions prior to referral, typically strategies that are generic and require little individualized planning to implement. There were no reported strategies that connoted tangible changes in curriculum, instruction, or management, features that we know are necessary for appropriate interventions. When we think about these findings, and compare them to the knowledge base we have accumulated, we find that we have very little information to which to compare our data. It seems reasonable to think that if we knew more about the types of students being referred, we might be able to do a better job of targeting prereferral activities. More important, we need to know why so many students are being referred repeatedly, and what this “cycling” has to do with the structure and functioning of the team and the nature of the problems themselves. We need to know whether or not we can connect 164 referral types and intervention types, and what the outcomes of specific interventions are for specific types of students. In short, extensive additional research is needed in terms of types of referrals to prereferral intervention programs. 165 REFERENCES 166 References Adelman, H. S., & Taylor, L. (1998). Involving teachers in collaborative efforts to better address the barriers to student learning. Preventing School Failure, 42(2), 55-60. Babbie, E. (1989). The practice of social research. Belmont, CA: Wadsworth. Bahr, M. W. (1994). The status and impact of prereferral intervention: "We need a better way to determine success". Psychology in the Schools, 31, 309-318. Bahr, M. W., Fuchs, D., Fuchs, L. S., & Fernstrom, P. S., P. (1993). Effectiveness of students versus teacher monitoring during prereferral intervention. Exceptionality, 4, 17-30. Bahr, M. W., Whitten, E., & Dicker, L. (1999). A comparison of school-based intervention teams: implications for educational and legal reform. Exceptional Children, 66(1), 68 - 73. Bay, M., Bryan, T., & O'Connor, R. (1994). Teachers assisting teachers: A prereferral model for urban educators. Teacher Education and Special Education, 1 7(1), 10- 2]. Beck, R. (1991). Project RIDE. Teaching Exceptional Children, 23(2), 60 - 61. Bogdan, R. C ., & Biklen, S. K. (1998). Qualitative Research in Education: An Introduction to Theory and Methods. (3rd ed.). Needham Heights, MA: Allyn & Bacon. Borg, W. R., Gall, I. P., & Gall, M. D. (1993). Applying Educational Research: A Practical Guide. (Third Edition ed.) White Plains, NY: Longman. Brown, J., Gable, R. A., Hendrickson, I. M., & Algozzine, B. (1991). Prereferral intervention practices of regular classroom teachers. Implications for regular and special education preparation. Teacher Education and Special Education, 14(3), 192-197. Bruskewitz, R. (1998). Collaborative intervention: A system of support for teachers attempting to meet the needs of students with challenging behaviors. Preventing School Failure, 42(3), 129-134. Burstein, N. D., & Sears, S. (1998). Preparing on-the-job teachers for urban schools: implications for teacher training. Teacher Education and Special Education, 21(1), 47-62. Carter, 1., & Sugai, G. (1989). Survey on prereferral practices: Responses from state departments of education. Exceptional Children, 55, 298-302. 167 Chalfant, J., C., Pysh, M., Miros, R., Bradshaw, E., & Adams, S. (1991). Teacher assistance teams: Supporting at-risk students in rural areas. A three year plan. Paper presented at the Rural Education Symposium. Chalfant, J. C., & Pysh, M. V. (1989). Teacher Assistance Teams: Five descriptive studies on 96 teams. Remedial and Special Education, 10(6), 49-58. Chalfant, I. C., Pysh, M. V., & Moultrie, R. (1979). Teacher Assistance Teams: A model for within building problem solving. Learning Disability Quarterly, 2, 85-89. Cohen, S. S., Thomas, C. C., Sattler, R. O., & Morsink, C. V. (1997). Meeting the challenge of consultation and collaboration: Developing interactive teams. Journal of Learning Disabilities, 30(4), 427-432. Conoley, J. C ., & Conoley, C. W. (1982). School Consultation: A Guide to Practice and Training. (Vol. 111). New York: Pergamon Press. Ferguson, D. L. (1995). The real challenge of inclusion: Confessions of a "Rabid inclusionist". Phi Delta Kappan, 77, 281-287. Flugum, K. R., & Reschly, D. J. (1994). Prereferral interventions: Quality indices and outcomes. Journal of School Psychology, 3(1), 1-14. Fraenkel, J. R., & Wallen, N. E. (1996). How to Design and Evaluate Research in Education. (Third ed.) New York: McGraw-Hill, Inc. Fuchs, D. (1987). Prereferral intervention for diflicult-to-teach students: Mainstream Assistance Teams - Years 1 and 2 (ERIC Document 293275). Fuchs, D., & Fuchs, L. S. (1989). Exploring effective and efficient prereferral interventions: A component analysis of behavioral consultation. School Psychology Review, 18(2), 260-278. Fuchs, D., & Fuchs, L. S. (1996). Consultation as a technology and the politics of school reform. Remedial and Special Education, 1 7(6), 386-3 92. Fuchs, D., Fuchs, L. S., Bahr, M., Reeder, P., Gilman, S., Fernstrom, P., & Roberts, H. (1990a). Prereferral intervention to increase attention and work productivity among difficult-to-teach pupils. Focus on Exceptional Children, 22(6), 1-8. Fuchs, D., Fuchs, L. S., & Bahr, M. W. (1990b). Mainstream Assistance Teams: A scientific basis for the art of consultation. Exceptional (..‘hildren. 57(2), 128-139. Fuchs, D., Fuchs, L. S., Bahr, M. W., Fernstrom, P., & Stecker, P. M. (1990c). Prereferral intervention: A prescriptive approach. Exceptional Children, 56(6), 493-513. 168 Fuchs, D., Fuchs, L. S., Gilman, S., Reeder, P., Bahr, M., Femstrom, P., & Roberts, H. (1990d). Prereferral intervention through teacher consultation: Mainstream Assistance Teams. Academic Therapy, 25(3), 263-276. Graden, J. L. (1989). Redefining "prereferral" intervention as intervention assistance: Collaboration between general and special education. Exceptional Children, 56(3), 227-231. Graden, J. L., Casey, A., & Bonstrom, O. (1983). Prereferral interventions: Eflects on referral rates and teacher attitudes (244438). Minneapolis: Institute for Research on Learning Disabilities, University of Minnesota. Graden, J. L., Casey, A., & Bonstrom, O. (I985a). Implementing a prereferral intervention system: Part II. The data. Exceptional Children, 56(3), 227-231. Graden, J. L., Casey, A., & Christenson, S. L. (I985b). Implementing a prereferral intervention system: Part I. The model. Exceptional C hildren. 51(5), 377-3 84. Harrington, R. G., & Gibson, E. (1986). Preassessment procedures for learning disabled children. Journal of Learning Disabilities, 19, 53 8-54 1. Hartman, W. T., & Fay, T. A. (1996). Cost-effectiveness of instructional support teams in Pennsylvania . Palo Alto, CA: American Institutes for Research in the Behavioral Sciences. Henry, G. T. (1990). Practical Sampling. (Vol. 21). Newbury Park: Sage Publications. Heron, T. E., & Harris, K. C. (1987). The Educational Consultant. Austin, TX: Proed. Hosmer, D. W., & Lemethow, S. (1989). Applied Logistic Regression. New York: John Wiley & Sons. IDEA. (I997). Individuals with Disabilities Education ActAmendments. H. R 5. (Vol. Ist Sess.) Idol, L. (1993). Special Educator's Consultation Handbook. (Second ed.) Austin, Texas: Pro-ed. Idol, L. (1997). Key questions related to building collaborative and inclusive schools. Journal of Learning Disabilities, 30(July/August), 384-3 94. Idol, L., Paolucci-Whitcomb, P., & Nevin, A. (1986). Collaborative Consultation. Rockville, Maryland: Aspen Publishers, Inc. Idol-Maestas, L. (1983). Special Educator’s Consultation Handbook. Rockville, Maryland: Aspen Systems Corporation. 169 Inman, L., & Tollefson, N. (1988). Elementary teachers’ attitudes toward preassessment procedures. Psychology in the Schools, 25, 331-337. Johnson, L. J ., & Pugach, M. C. (1990). Classroom teachers' views of intervention strategies for learning and behavior problems: Which are reasonable and how frequently are they used? The Journal of Special Education, 24(1), 69-84. Johnson, L. J., & Pugach. M. C. (1991). Peer Collaboration: Accommodating students with mild learning and behavior problems. Exceptional C hildren. 5 7(5), 454-461. Jordan, A. (1994). Skills in Collaborative Classroom Consultation. London: Routledge. Kahl, L. J ., & Fine, M. J. (1978). Teachers' perceptions of the school psychologist as a function of teaching experience, amount of contact, and socioeconomic status of the school. Psychology in the Schools, 15(4), 577 - 582. Knoff, H. M., Hines, C. V., & Kromrey, J. D. (I995a). Finalizing the consultant effectiveness scale: An analysis and validation of the characteristics of effective consultants. School Psychology Review, 24(3), 480-496. Knoff, H. M., Sullivan, P., & Liu, D. (1995b). Teacher's ratings of effective school psychology consultants: An exploratory factor analysis study. Journal of School Psychology, 33(1), 39-57. Korinek, L., & McLaughlin, V. (1996). Preservice preparation for interdisciplinary collaboration: The intervention assistance teaming project. Contemporary Education, 68, 41-44. Kovaleski, J. F., Gickling, E. E., & Morrow, H. (1999). High versus low implementation of instructional support teams: a case for maintaining program fidelity. Remedial and Special Education, 20(3), 170-183. Kovaleski, J. F ., Tucker, J. A., & Stevens, L. J. (1996). Bridging special and regular education: The Pennsylvania initiative. Educational Leadership, 53(5), 44-47. Kutsick, K. A., Gutkin, T. B., & Witt, J. C. (1991). The impact of treatment development process, intervention type, and problem severity on treatment acceptability as judged by classroom teachers. Psychology in the Schools, 28. Long, T. J ., Convey, J. J., & Chwalek, A. R. (1985). Completing Dissertations in the Behavioral Sciences and Education. San Francisco: J ossey-Bass Publishers. Mann, P. (1973 ). Student consultants: Evaluations by consultees. American Journal of Community Psychology, 1, 182 - 193. Mortenson, B. P., & Witt, J. C. (1998). The use of weekly performance feedback to increase teacher implementation of a prereferral academic intervention. The School Psychology Review, 27(4), 613-627. 170 Murdick, N. L., & Petch-Hogan, B. (1996). Inclusive classroom management: Using preintervention strategies. Intervention in School and Clinic, 31(3), 172-176. Nelson, J. R. (1991). Prereferral intervention: A review of the research. Education and Treatment of Children, 14(3), 243-253. Nelson, J. R., Smith, D. J ., Taylor, L., Dodd, J. M., & Reavis, K. (1992). A statewide survey of special education administrators regarding mandated prereferral interventions. Remedial and Special Education, 13(4), 34-3 9. Noell, G. H., & Witt, J. C. ( 1999). When does consultation lead to intervention implementation? Critical issues for research and practice. The Journal of Special Education, 33(1), 29-35. Noell, G. H., Witt, J. C., Gilbertson, D. N., Ranier, D. D., & Freeland, J. T. (1997). Fundamental agreements and epistemological differences in differentiating what was said from what was done in behavioral consultation. School Psychology Quarterly, 13, 81-88. Nyikos, M., & Hashimoto, R. (1997). Constructivist theory applied to collaborative learning in teacher education: In search of ZPD. The Modern Language Journal, 81(Winter), 506-517. Ohio State Department of Education. (1985). Intervention Assistance Teams . Columbus: Division of Elementary and Secondary Education. Payne, S. (1999). Interview in Qualitative Research. In A. Memon & R. Bull (Eds), Handbook of the Psychology of Interviewing . Chichester, England: John Wiley & Sons Ltd. Pobst, A., & Patriarca, L. A. (1999). Comparing teacher and psychologist perspectives: Reasonability and use of prereferral interventions. Manuscript in progress, Michigan State University. Pugach, M. C ., & Johnson, L. J. (1988). Peer Collaboration. Teaching Exceptional Children, 20(3), 75-77. Pugach, M. C., & Johnson, L. J. (1989). The challenge of implementing collaboration between general and special education. Exceptional Children, 56(3), 232-235. Pugach, M. C ., & Johnson, L. J. (1995). Unlocking expertise among classroom teachers through structured dialogue: Extending research on peer collaboration. Exceptional Children, 62(2), 101-110. Pugach, M. C., & Warger, C. L. (1995). Curriculum Considerations. In J. I. Goodland & T. C. Lovitt (Eds), Integrating General and Special Education (pp. 125-148). New York: Macmillan. 171 Rosenfield, S. (1992). Developing school-based consultation teams. A design for organizational change. School Psychology Quarterly, 7(1), 27-46. Rosenfield, S. A., & Gravois, T. A. (1996). Instructional Consultation Teams: Collaborating for Change. New York: The Guilford Press. Schumm, J. S., Vaughn, 5., Gordon, J., & Rothlein, L. (1997). General education teachers' beliefs, skills, and practices in planning for mainstreamed students with learning disabilities. Teacher Education and Special Education, 1 7(1), 22-3 7. Short, R. J., & Talley, R. S. (1996). Effects of teacher assistance teams on special education referrals in elementary schools. Psychological Reports. 79(1431-1438). Stenger, M. K., Tollefson, N., & Fine, M. J. (1992). Variables that distinguish elementary teachers who participate in school-based consultation from those who do not. School Psychology Quarterly, 7(4), 271-284. Sugai, G. M., & Tindal, G. A. (1993). Eflective School Consultation: An Interactive Approach. Pacific Grove, California: Brooks/Cole Publishing Company, a division of Wadsworth, Inc. Telzrow, C. F. (1997). Best practices in facilitating intervention adherence. In A. Thomas & J. Grimes (Eds), Best Practices in School Psychology - III . Bethesda, MD: National Association of School Psychologists. Tharp, R. G. (1975). The triadic model of consultation: Current considerations. In C. A. Parker (Ed. ), Psychological Consultation: Helping Teachers Meet Special Needs . Reston, VA: The Council for Exceptional Children. Vaughn, 8., Schumm, J. S., Jallad, B., Slushcr, J ., & Saumell, L. (1996). Teachers' views of inclusion. Learning Disabilities Research and Practice, 11(2), 96- 106. West, J. F., & Idol, L. (1987). School consultation (Part 1): An interdisciplinary perspective on theory, models, and research. Journal of Learning Disabilities, 20(7), 388-408. Whitfield, T. J. (1996). Academic eflectiveness of Allegheny County middle schools' instructional support teams. (ED 560). Whitten, E., & Dicker, L. (1995). Intervention assistance teams: A broader vision. Preventing School F ailure, 40(1), 41-45. Yin, R. K. (1994). Case study research: design and methods. (Second ed.) (Vol. 5). Thousand Oaks: SAGE Publications. Zins, J. E., & Ponti, C. R. ( I996). The influence of direct training in problem solving on consultee problem clarification skills and attributions. Remedial and Special Education, 1 7(Novembcr), 370-3 76. 172 u. J ll?!” 1”] 111 3 1293 0209