l-.oII. a-- A UNIVERSITY LIBRARIE Ilillllll’mlfl l‘ “mm 09 3'42. 80%,; mm ll , 129300786 4378 3 LIT”? ’ ”Y W Michigan State University yl “—7 This is to certify that the dissertation entitled A GENERALIZABILITY ANALYSIS OF OBSERVATIONAL ABILITIES IN THE ASSESSMENT OF HOPPING USING TWO DEVELOPMENTAL APPROACHES TO MOTOR SKILL SEQUENCING presented by Mary Ann Painter has been accepted towards fulfillment of the requirements for Ph . D . degree in Exercise Science and Physical Education Major professor Date f/i/f9 / / , MSU is an Affirmative Action/Equal Opportunity Institution 0-12771 PLACE N RETURN BOX to remove this checkout from your record. 1’0 AVO. FINES return on or baton duo duo. DATE DUE DATE DUE DATE DUE MSU Io An Animdlvo AM“ Opportunity Institution own-”5W A GENERALIZABILITY ANALYSIS OF OBSERVATIONAL ABILITIES IN THE ASSESSMENT OF HOPPING USING TWO DEVELOPMENTAL APPROACHES TO MOTOR SKILL SEQUENCING BY Mary Ann Painter A DISSERTATION Submitted to Michigan State University in partial fulfillment of the requirements for the degree of DOCTOR OF PHILOSOPHY School of Health Education, Counseling Psychology and Human Performance 1989 ‘IIU‘tl‘q ABSTRACT A GENERALIZABILITY ANALYSIS OF OBSERVATIONAL ABILITIES IN THE ASSESSMENT OF HOPPING USING TWO DEVELOPMENTAL APPROACHES TO MOTOR SKILL SEQUENCING BY Mary Ann Painter Elementary school personnel responsible for the physical education of children must be able to observe and analyze motor skill behavior. Intratask developmental motor sequences provide a potential observational system by which the assessment of developing motor patterns can be facilitated. Both body-part sequences and total-body sequences exist for a number of fundamental motor skills, including the skill of hopping. This research project was designed to investigate the generalizability with which both upper-division, undergraduate, kinesiology and elementary education students can rate the arm action, leg action, and total-body action of children's hopping performances according to pre— longitudinally validated developmental sequences. The effects of training observers to use developmental sequences as assessment criteria and the minimal conditions of measurement required to achieve acceptable levels (.80) of generalizability were also examined. Mary Ann Painter Five videotaped trials of twenty boys and girls between 3.5 and 8.5 years of age were rated by twenty observers. These observers were assigned to one of four observational groups: (a) kinesiology students using a totalebody developmental sequence; (b) kinesiology students using arm and leg developmental sequences: (c) elementary education students using a total-body developmental sequence; and (d) elementary education students using arm and leg developmental sequences. A Qgfieralized Analysis Qf Egriance was conducted on the data. The findings indicate a trend for the reduction of measurement error in the assessment of hopping when kinesiology students and elementary education students are trained in the used of developmental sequences as assessment instruments. The kinesiology students were more consistent than the elementary education students in employing the arm and leg component sequences, whereas the elementary education students were more consistent in their total-body ratings. For both categories of observers, the leg sequence was found to be reliably employed with fewer observers and/or trials than either the arm sequence or the total-body sequence. The kinesiology observers also were able to use the arm sequence more reliability (with fewer observers) Mary Ann Painter than the total~body sequence. For the elementary education observers, the minimal conditions of measurement for the total-body sequence and the arm sequence differed by only one trial. DEDICATION There is nothing so fine as a community of friends that is there when you need them, that helps you through the bad times, and plays with you in the good times, that encourages you to be yourself. I began to know such a community during my first months in East Lansing, and my friendships with these people have grown stronger with each passing year. It is this community of friends that has stood behind me, and has given me encouragement and support over the last six years. For all of these individuals I have the utmost warmth and appreciation. In particular, there are those individuals to whom I owe special thanks to whom I dedicate this dissertation. Irene Blanchard, Jon, Nathan, & Benjamin Pumplin David and Lizzie Kanistanaux Bob and Laura Stein Abby Schwartz John Bedigian and, of course, THE BALKAN BAND (Lansingskite Gusteri Balkan Orkester) ACKNOWLEDGEMENTS A doctoral program and a project of this nature are not undertaken without the nurturing, encouragement, and assistance of dedicated faculty, colleagues, and friends. To these individuals I express my sincere gratitude. I wish to extend special thanks to my guidance committee for their dedication and patience during the weeks prior to my dissertation defense. To Dr. Crystal Branta, my advisor and committee chair, I extend my utmost appreciation for her continuing faith in my capabilities, as well as her encouragement and guidance throughout my program. I particularly thank Crystal for the time and support she gave to me during the final weeks in the preparation of this dissertation. My thanks is also extended to Dr. John Haubenstricker for his valuable advice and guidance during my years as a student and graduate research assistant. His high standards and careful editing were appreciated and will continue to be an inspiration in my future endeavors. I wish to thank Dr. Marty Ewing for her countless hours ‘of consultation on generalizability theory, as well as her compassion for the trials and tribulations of graduate school. I also extend my gratitude to Dr. Hiram Fitzgerald vi for sharing his expertise during the preparation of this dissertation, and for inspiring me to question and to research developmental theories. The participants "behind the scene" in the collection of my data are numerous. To the following individuals, I am extremely grateful: the children enrolled in the summer programs at Michigan State University and at the Lansing Y.M.C.A.; Paul Behen, Cathy Lirgg, and Deb Kiefuik for their assistance in videotaping the children; Dr. Marjorie Kostelnik, Marilyn Waters, Duane Whitbeck, and Laura Stein for their cooperation in organizing the children at the Child Development Laboratories; Monica Chapin for organizing the children in the Early Childhood Motor Skills Project; Carrie Fitzgerald, Darcy Hunt, and Wade Hamptom for promoting this project at the Y.M.C.A.; the kinesiology and elementary education students at the University of Colorado for their interest and participation in this study; Dr. Dale Ulrich, Dr. Dave Solomon, and Dr. Bob Floden for their advice and consultation on generalizability theory: and Clersida Garcia for serving as my on-campus courier. Finally, three individuals deserve special thanks for their support and encouragement: 'Judy Kretschmar for her countless hours of discussion and contemplation; Dr. Penny McCullagh for her enthusiasm and friendship, as well as her video equipment; and my loving sister, Toni Christine Painter, for her never-ending patience and confidence in me. vii TABLE OF CONTENTS LIST OF TABLES ......................................... x LIST OF FIGURES ........................................ xii CHAPTER I: INTRODUCTION ............................... 1 Need for the Study ................................ 6 Purpose of the Study ........... ' ................... 9 Terminology ..... .................................. 12 Scope of the Study ................................ 15 Research Hypotheses ............................... 16 Assumptions ....................................... 17 Limitations .. ..................................... 17 CHAPTER II: REVIEW OF LITERATURE ...................... 19 Descriptive Research in Children's Motor Skill Achievements .......................... 20 1920 - 1940: The Foundation Years in the Study of Motor Development ....... 22 1940 - 1970: The Quantitative Description of Children's Motor Achievements ........ 27 1940 - 1970: The Qualitative Description of Children's Motor Achievements ........ 30 1970 - 1988: Current Trends in Motor Development Research .............. 32 Stage Theories of Development ... .................. 33 Piaget's Criteria for Developmental Stages ... 36 Developmental Sequences vs. Developmental Stages ..... . .............. 45 Current Application of Developmental Sequences to Fundamental Motor Skills ....................... 48 Criteria for Developmental Motor Sequences ... 50 Conceptualizing Developmental Motor Sequences ........................ 56 Prelongitudinal Screening Methods for the Study of Developmental Motor Sequences .. 62 viii Observational Skills As Related to Motor Development ...... . ................................ 67 Observation in the Instructional Process ..... 69 The Skill of Observation ........... . ..... .... 71 The Relationship of Kinesthetic Experiences and Teaching Experiences to Observational Skill .................. 73 The Relationship of Training Specificity to Observational Skill ... ............... 74 Research Questions in the Development of Observation Systems .................. 78 Generalizability Theory ........................... 80 Terminology .................................. 86 Framework for Conducting a Generalizability Analysis ............. 90 Application of Generalizability Analysis to Motor Performance Assessment ......... 93 The Development of Hopping ........................ 97 Defining a Hop ....... ...... .................. 99 Descriptive Research on Hepping .............. 101 A Composite Developmental Sequence for Hopping ............................. 104 Component Developmental Sequences for Hopping ............................. 113 The Developmental Ages of Hopping ............ 118 Summary ........................................... 123 CHAPTER III: METHODOLOGY .............................. 125 Subjects .......................................... 126 Observers ............ ........................ ..... 128 Procedures ....................................... 131 Videotaping the Subjects Performing the Hopping Task ........................ 131 Editing the Videotape ........................ 134 Establishing the_Reliability and Objectivity of the Investigator ......... 136 Meeting with the Observers .. ................. 138 Obtaining Pre-training Ratings of the Data-Collection Videotape ........ 139 Conducting Training and Obtaining Post-training Ratings of the Data-Collection Videotape ............... 141 Measurement Design ................................ 142 Assumptions of Generalizability Analysis ..... 143 G- and D- Study Designs and ~Statistical Analyses .................... 145 ix CHAPTER IV: RESULTS AND DISCUSSION .................... 153 Investigator Agreement and Reliability ............ 154 Inter-Investigator Agreement and Generalizability .... .................... 155 Intra-Investigator Agreement and Generalizability ........................ 161 Analysis of Pre- and Post- Training Effects ....... 165 The Effects of Training on Using the Arm Sequence to Assess Hopping Ability .. 168 The Effects of Training on Using the Leg Sequence to Assess Hopping Ability .. 170 The Effects of Training on Using the Composite Sequence to Assess Hopping Ability ............. . ........... 172 Discussion ................................... 174 Analysis of the Developmental Sequences By Observational Groups ........................... 178 D-Study Optimization and Classifications of Generalizability Coefficients ........ 179 Arm Sequence Analysis ........................ 182 Leg Sequence Analysis ........................ 190 Composite Sequence Analysis .................. 198 Comparison of the Analyses of the Three Development Sequences ............. 208 Discussion ..... . ............................. 213 CHAPTER V: SUMMARY, CONCLUSIONS. PRACTICAL IMPLICATIONS, AND RECOMMENDATIONS ......... 223 Conclusions . ...................................... 230 Practical Implications ............................ 232 Recommendations ................................... 235 APPENDIX A: Subjects' Informed Consent ..... .......... 239 APPENDIX B: Observers' Informed Consent .............. 242 APPENDIX C: Page 1 of Observer Scoresheets ........... 245 APPENDIX D: Raw Data ................................. 248 APPENDIX E: G-Study Analysis of Variance Tables ...... 254 BIBLIOGRAPHY ........................................... 263 10. 11. LIST OF TABLES Percent of Children Hopping on the Preferred Foot at Various Stages ............................ 108 Percent of Children Hopping on the Non-Preferred Foot at Various Stages .............. 109 Mean Squares and Variance Estimates for Inter-Investigator Generalizability (Arm Sequence) .................................... 156 Mean Squares and Variance Estimates for Inter-Investigator Generalizability (Leg Sequence) .... .......... . ..................... 156 Mean Squares and Variance Estimates for Inter-Investigator Generalizability (Composite Sequence) .............................. 157 Inter-Investigator Generalizability Coefficients and Percent Agreement Scores ...................... 160 Mean Squares and Variance Estimates for Intra-Investigator Generalizability (Arm Sequence) .................................... 162 Mean Squares and Variance Estimates for Intra-Investigator Generalizability (Leg Sequence) .................................... 162 Mean Squares and Variance Estimates for Intra-Investigator Generalizability (Composite Sequence) .............................. 163 Intra-Investigator Generalizability Coefficients and Percent Agreement Scores ...................... 163 Variance Estimates and Percent Variance for Pre- and Post- Training Generalizability (Arm Sequence) .................................... 169 xi 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23. 24. Variance Estimates and Percent Variance for Pre- and Post- Training Generalizability (Leg Sequence) ... ........................ . ........ Variance Estimates and Percent Variance for Pre- and Post- Training Generalizability (Composite Sequence) .............................. Variance Estimates and Percent Variance for Elementary Education Observers (Arm Sequence) ..... Variance Estimates and Percent Variance for Kinesiology Observers (Arm Sequence) .............. Subject Score Generalizability Coefficients Pre- and Post- Training for All Observers (Arm Sequence) ...... . ............................. Subject Score, Inter-Observer, and Inter-Trial Generalizability Coefficients for All Observers (Arm Sequence) .................. ...... ... ......... Variance Estimates and Percent Variance for Elementary Education Observers (Leg Sequence) ..... Variance Estimates and Percent Variance for Kinesiology Observers (Leg Sequence) . ............. Subject Score Generalizability Coefficients Pre- and Post- Training for All Observers (Leg Sequence) .................................... Subject Score, Inter-Observer, and Inter—Trial Generalizability Coefficients for All Observers (Leg Sequence) .... ................................ Variance Estimates and Percent Variance for Elementary Education Observers (Composite Sequence) .............................. Variance Estimates and Percent Variance for Kinesiology Observers (Composite Sequence) ........ Subject Score Generalizability Coefficients Pre- and Post- Training for All Observers (Composite Sequence) .............................. xii 25. 26. 27. 28. 29. 30. 31. 32. 33. 34. 35. 36. 37. 38. Subject Score, Inter-Observer, and Inter-Trial eneralizability Coefficients for All Observers (Composite Sequence) ........ .... ....... ..... ...... 205 Post-Training Generalizability Coefficients For All Observational Groups ...................... 209 Percent of Total Variance Attributable to All Facets in All Post-Training Analyses .............. 210 Analysis of Variance Table for Pre-Training Ratings (All Observers Using Arm Sequence) ........ 254 Analysis of Variance Table for Post-Training Ratings (All Observers Using Arm Sequence) ........ 254 Analysis of Variance Table for Pre-Training Ratings (All Observers Using Leg Sequence) ........ 255 Analysis of Variance Table for Post-Training Ratings (All Observers Using Leg Sequence) ........ 255 Analysis of Variance Table for Pre-Training Ratings (All Observers Using Total-Body Sequence) . 256 Analysis of Variance Table for Post-Training Ratings (All Observers Using Total-Body Sequence) . 256 Analysis of Variance Table for Pre-Training Ratings (Elementary Education Observers Using Arm Sequence) ............................... 257 Analysis of Variance Table for Post-Training Ratings (Elementary Education Observers Using Arm Sequence) ............................... 257 Analysis of Variance Table for Pre-Training Ratings (Kinesiology Observers , Using Arm Sequence) ............................... 258 Analysis of Variance Table for Post-Training Ratings (Rinesiology Observers Using Arm Sequence) ...... ...... . .................. 258 Analysis of Variance Table for Pre-Training Ratings (Elementary Education Observers Using Leg Sequence) ... ............................ 259 xiii 39. 40. 41. 42. 43. 44. 45. Analysis of Variance Table for Post-Training Ratings (Elementary Education Observers Using Leg Sequence) .......... . ........ .. ....... Analysis of Variance Table for Pre-Training Ratings (Kinesiology Observers Using Leg Sequence) ............................ Analysis of Variance Table for Post-Training Ratings (Kinesiology Observers Using Leg Sequence) ............................ Analysis of Variance Table for Pre-Training Ratings (Elementary Education Observers Using Total-Body Sequence) ..................... Analysis of Variance Table for Post-Training Ratings (Elementary Education Observers Using Total-Body Sequence) ..................... Analysis of Variance Table for Pre-Training Ratings (Kinesiology Observers Using Total-Body Sequence) ..................... Analysis of Variance Table for Post-Training Ratings (Kinesiology Observers Using Total-Body Sequence) ..................... xiv LIST OF FIGURES Observed Frequencies of Occurrence for Males of Hopping Stages Across Age ..................... Observed Frequencies of Occurrence for Females of Hopping Stages Across Age ..................... Film Tracings of the Developmental Sequence of Hopping as Filmed by Seefeldt and Haubenstricker Floor Plan for Videotaping Sessions .............. Linear Model and Venn Diagram for G Study Design 1: s x t x (ozc) ...... . .................. Linear Model and Venn Diagram for G Study Design 2: s x o x t ............................. XV CHAPTER I INTRODUCTION The developmental changes occurring in children's fundamental movement patterns have been recorded both quantitatively and qualitatively for the better part of this century by researchers interested in child development, in motor development, and in physical change. Hohlwill (1973) states, "...whenever we observe the changes ‘produced' by development, we are recording the outcome of a natural process....The results will not be as neatly interpretable as they are where the experimenter manipulates a particular variable to produce the changes, but we will need to have an accurate picture of them so that we may know exactly what it is that we need to explain through further theory and experiment” (p. 159). Researchers who observe developmental changes in movement patterns often describe these changes according to some theoretical or philosophical scheme. This descriptive scheme ultimately presents itself as the dependent variable from which researchers then can draw inferences can about developmental processes in human beings. Changes in movement patterns also must be observable by teachers who wish to influence the development of motor skills in children. In this more practical context, a researcher's scheme serves as the measurement instrument by which teachers draw inferences about developmental processes in their students. Developmental changes in motor patterns have been described from a number of perspectives. Hickstrom (1983) distinguished between trends and stages in skill development. Trends indicate the general course of change, whereas stages represent more step-like interpretations of development. Phases, levels, steps, and stages are terms that have all been used to identify qualitatively different behaviors in changing motor patterns. When the research in motor development is examined carefully, it is evident that there are differences not only in what is observed and how it is observed, but also in the philosophies behind the observational schemes. The description of children's developing movement has been concerned with the order of occurrence of motor patterns (Bayley, 1935: Cunningham, 1927: Gesell & Amatruda, 1964: McCaskill E Hellman, 1938). the proficiency of motor patterns (Deach, 1951: Gutteridge, 1939: Gesell, 1929, 1939, 1940: Halverson, 1932: Hellebrandt, Rarick, Glassow, & Carns, 1961: McGraw, 1943, 1946; Sinclair, 1974; Shirley, 1931), interskill, intraskill, and across skill patterns (Roberton, 1978c, 1982: Seefeldt & Haubenstricker, 1982). and total-body descriptions verses body-part descriptions (Roberton, 1978c, 1982: Seefeldt a Haubenstricker, 1982). What emerges from all approaches in the systematic study of motor behavior is that the development of motor abilities is sequential and predictable (Haubenstricker & Seefeldt, 1976: Branta, Haubenstricker, G Seefeldt, 1984). The focus of motor development research during the past two decades has been on the description of changes in developing movement patterns, rather than on age-specific scales of development or product scores of motor achievement. Intratask developmental motor sequences for fundamental motor skills have become an important heuristic model for both research and teaching (Roberton, 1977a). The changing configurations of the body as it progresses in mechanical efficiency from immature patterns of movement to mature patterns have been described for walking, running, jumping, hopping, galloping, skipping, throwing, catching, kicking, punting, striking, dribbling, and rolling (Haubenstricker & Seefeldt, 1986: Williams, 1980). The impetus behind research describing developmental motor sequences was the recognition that teachers should know, understand, and be able to identify the movement patterns of their students (Halverson, 1971: Halverson, Roberton, E Harper, 1973: Roberton & Halverson, 1984: Seefeldt, Reuschlein, E Vogel, 1972: Seefeldt, 1980: Hickstrom, 1983). Such knowledge is useful in developing formal and informal assessment instruments by which to plan, implement, and evaluate intervention methods designed to facilitate the development of motor skills. Interestingly, the evaluation of students' progress in motor skill acquisition continues to occur primarily through product or achievement scores, rather than through skill analysis (Mosher a Schutz, 1983: Ulrich, Ulrich, & Branta, 1988). Mosher and Schutz (1983) have suggested that perhaps the information obtained during the past two decades has not been assembled into a usable observational system. The ability of teachers to assess motor performance is enhanced with the structured use of reliable, descriptive checklists (Bayless, 1981: Hoffman, 1977b: Taylor, 1979) and with appropriate training in skill analysis. Research has indicated that if undergraduate kinesiology and/or physical education majors plan a career in facilitating the motor skill acquisition of children and adults, they must be trained to observe and analyze movement as it relates to teaching and coaching (Allison, 1987: Armstrong & Hoffman, 1979: Barrett, 1979b: Hoffman, 1974, 1977a, 1983; Locke. 1972). Since classroom teachers often are responsible for physical education classes at the elementary and preschool levels, that they should also be taught to observe and analyze developing motor skills (Morrison 5 Reeve, 1986: Morrison, Reeve, 5 Harrison, 1984: Robinson, 1974). Barrett (1977) has stressed that observational skill is only as good as the support system on which it is based. Although there is great potential for developmental motor sequences to serve as assessment criteria in observational systems, Roberton (1977a, 1989) has noted that the accuracy with which teachers can identify developmental levels must first be ascertained. From this information, motor sequences then can be modified and incorporated into observational checklists. Two studies have been conducted that examine the reliability of observers using developmental sequences to rate children's motor performances (Mosher & Schutz 1983; Ulrich, Ulrich, & Branta, 1988). Both studies employed a generalizability analysis to determine the effect of changing measurement conditions on observer ability. The refinement of generalizability theory in recent years (Cronbach, Gleser, Nanda, Rajaratnam, 1972; Cardinet, Tourneur, & Allal, 1976, 1981; Rentz, 1980; Shavelson & Webb, 1981) has provided a statistical method by which to establish the reliability of measurement scales under changing conditions of measurement. With indices of both observer agreement and measurement generalizability available to the motor development researcher, an understanding of the usefulness of developmental sequences is now being investigated. Need for the Study Intratask developmental motor sequences for fundamental motor skills have been described from two major perspectives. Researchers at Michigan State University have identified total-body configurations in changing movement patterns (Branta, Haubenstricker, & Seefeldt, 1984; Seefeldt et al., 1972), while researchers at the University of Wisconsin prefer describing changing movement patterns in body-part configurations. Several of these motor sequences have been prelongitudinally validated (Branta, Haubenstricker, Riger, & Ulrich, 1984; Clark, 8 Phillips, 1985: Halverson & Williams, 1985: Haubenstricker, Branta, & Seefeldt, 1983: Haubenstricker, Seefeldt, & Branta, 1983: Haubenstricker, Branta, Seefeldt, Brakora, & Rigor, 1989; Langendorfer, 1987a: Roberton, Williams, & Langendorfer, 1980: Williams, 1980). As longitudinal validation of these motor sequences continues, an examination of observer accuracy and reliability is necessary in order to determine the usefulness of these sequences to the practitioner (Roberton, 1977a, 1989: Godbout & Schutz, 1983). Only two studies have investigated the reliability with which developmental sequences can be used to rate motor skill performance. Mosher and Schutz (1983) examined the generalizability of observers using component, or body-part, sequences in assessing overarm throwing. Ulrich et a1. (1988) utilized composite, or total-body, sequences to determine the generalizability of observers rating hopping, jumping, and running. Both studies employed videotape as the observational medium, an appropriate first step in conducting a "direct-systematic analysis" of both observer accuracy and reliability (Arend a Higgins, 1976: Roberton, 1989). Although investigators of component sequences and investigators of composite sequences both advocate their techniques as being the preferred method for identifying developmental motor levels (Roberton, 1977a: Seefeldt & Haubenstricker, 1982), there has been no research conducted to date which compares the usefulness of either method to the practitioner. When consideration is given to which technique may be used most effectively by the practitioner, three possible scenarios exist. It may be that (a) both methods can be reliably adopted for the assessment of developing motor skills, (b) one method is preferable, or (c) neither method can be reliably employed. The choice of the sequencing technique by a practitioner may depend on the conditions of measurement which can range from minimal (one observer and one trial) to numerous (several observers and several trials). In addition, the experiences of practitioners in observing and analyzing motor skills may affect their ability to use a particular sequencing method. It is also possible that the suitability of the sequencing technique will change with the fundamental skill being assessed. Both component and composite sequences exist for walking, running, jumping, hopping, galloping, skipping, catching, throwing, striking, kicking, and punting (Haubenstricker & Seefeldt, 1986). Prelongitudinal validation exists for both sequencing techniques in jumping, hopping, throwing, and striking (Clark & Phillips, 1985: Early Childhood Motor Skills Development Study, 1978-1984; Halverson E Williams, 1983: Haubenstricker et al., 1989: Langendorfer, 1987b; Roberton, 1978a: Roberton & Langendorfer, 1980). An investigation into the reliability with which any of these prelongitudinally validated sequences can be used is needed. Purpose of the Study The purpose of this study was threefold: (a) to analyze the effects of training observers to rate the hopping performance of children using developmental sequences as assessment instruments: (b) to investigate the generalizability with which upper-division, undergraduate kinesiology students and upper-division, undergraduate students pursuing teaching certification in elementary education can use composite and component developmental motor sequences to assess the hopping performance of children: and (c) to examine the minimal conditions of measurement required by kinesiology students and the minimal conditions of measurement required by elementary education students when using two component sequences to assess hopping and when using a composite sequence to assess hopping. The skill of hopping was chosen for this study because it is one of the four fundamental motor skills for which prelongitudinal validation exists for both composite and component developmental sequences. Hopping is an important antecedent skill in the attainment of many locomotor and manipulative sport and dance skills. Furthermore, hopping is a skill of interest to the study investigator. 10 Three developmental sequences describing the changing body configurations during the development of hopping skill were examined in this study. Two of these sequences were component sequences hypothesized and prelongitudinally validated by Halverson and Williams (1985). One of the component sequences describes the action of the arms during hopping and the other sequence describes the action of the legs. The third sequence was a composite (total-body) developmental sequence hypothesized by Haubenstricker, Henn, and Seefeldt (1975) and prelongitudinally validated by Haubenstricker, Branta, Seefeldt, Brakora, and Riger (1989). The specific questions addressed in this study were as follows: i 1. Is there an improvement in the observational abilities of upper-division, undergraduate kinesiology students following training when using either a total-body developmental sequence, an arm-action deve10pmental sequence, or a leg-action developmental sequence to categorize the hopping performance of children? 2. To what degree of generalizability can upper- division, undergraduate kinesiology students assess the hopping performance of children using a total-body sequence, an arm-action sequence, and a leg-action sequence? 11 Is there an improvement in the observational abilities of upper-division, undergraduate elementary education students following training when using either a total-body developmental sequence, an arm-action developmental sequence, or a leg-action developmental sequence to categorize the hopping performance of children? To what degree of generalizability can upper- division, undergraduate elementary education students assess the hopping performance of children using a total-body sequence, an arm- action sequence, and a leg-action sequence? In order to assess children's hopping performance reliably, what are the minimal conditions of measurement that would be required by upper- division, undergraduate kinesiology students for each development sequence? In order to assess children's hopping performance reliably, what are the minimal conditions of measurement that would be required by upper- division, undergraduate elementary education students for each development sequence? Is there a difference in the minimal conditions of measurement required by either category of observers? 12 8. Is there a difference in the minimal conditions of measurement obtained for each of the three sequences? Terminology The following terms are presented to assist in the understanding of this study. Rinesiology. Part of the observers in this study were enrolled in a department of kinesiology. Rinesiology is the study of the art and science of human movement. As the base of knowledge in human movement increases, many traditional departments of physical education have expanded and diversified their programs of study. These changes are reflected in discipline-oriented departments, as opposed to education-oriented departments. Students interested in teaching physical education often major in kinesiology and obtain pedagogical training through departments of education. The alternative emphases for those students in a kinesiology major who are not interested in a teaching certification include such options as corporate fitness, cardiac rehabilitation, exercise physiology, sport psychology, sport and nutrition, and physical therapy. Despite the professional orientation of kinesiology majors, most programs of study in kinesiology 13 include at least one course on the developmental aspects of movement behavior and several courses concerned with analyzing the patterns of human movement. Elementary Education. Part of the observers for this study are referred to as elementary education students. It should be noted, however, that programs of education at many universities across the country have begun requiring students interested in teaching careers to obtain an undergraduate degree in a discipline other than the field of education. Pedagogical training directed toward the acquisition of teaching certification then becomes an area of study beyond a student's major. The elementary education observers in this study were upper-division, undergraduate students pursuing certification in elementary education, but majoring in subjects such as child psychology, communications, linguistics, and anthropology. Developmental Motor Sequence. In the study of motor development, a sequence is defined as a series of movements which is highly predictable with reference to the performer and the skill. For intraskill sequences, this series of movements represents the changes in body configuration that occur as an individual progresses from immature performance to mature performance in a single skill. Component Motor Sequence. When the steps in a developmental motor sequence describe the changing 14 configurations of a specific body part, such as the movement of the arms as distinguished from the movement of the legs or the trunk, then the sequence is referred to as a component sequence. - Composite Motor Sequence. When the steps in a developmental motor sequence describe the changing configurations of the total body, then the sequence is referred to as a composite sequence. Stqqg. The identifiable configurations in the steps of a developmental motor sequence are often referred to as stages. A review of the developmental psychology literature and the motor development literature reveals some controversy over the use of this term. It remains, however, a term that expresses levels of development in a motor sequence. Generalizability. An extension of the classical definition of reliability, generalizability is concerned with the consistency of scores, or measurements, across a universe of conditions under which the scores were obtained. The theoretical extension of classical test theory is known as generalizability theory. Terminology specific to generalizability theory is addressed further in Chapter 2. 15 Scope of the Study Fifty-nine children between the ages of 4 and 8 years were videotaped hopping a distance of fifteen feet. Each child was given one warm-up trial and five videotaped trials, all on the preferred foot. Twenty subjects were then randomly selected from an age-stratified pool of videotaped subjects and their performances were randomly consolidated on a data-collection videotape. One training videotape with six distinct training segments was produced from a portion of the remaining pool of subjects. Ten undergraduate kinesiology students and ten undergraduate elementary education students served as observers for this study. Five observers from each category were trained to rate the hopping performance of children using a composite sequence for hopping (Haubenstricker, Henn, & Seefeldt, 1975) and the other five observers were trained to use two component sequences for hopping, one describing the arm action and one describing the leg action (Halverson & Williams, 1983). Both prior to and following training, the observers rated the hopping performances of the children on the data-collection videotape. A generalizability analysis was conducted on the results obtained. 16 Research Hypotheses It was hypothesized that: 1. Training undergraduate students to use developmental sequences to rate the hopping performance of children would result in reduced measurement error for all observers using assigned sequences. When given equivalent training in the analysis of the development of hopping, there would be little difference in the generalizability with which kinesiology students and elementary education students rate the hopping performance of children regardless of the sequencing technique employed. The minimal conditions of measurement required to obtain generalizability of .80 would be more extensive for the total-body sequence than they would be for either the arm sequence or the leg sequence. 17 Assumptions This study was based on the following assumptions: 1. Hopping development occurs in an orderly and sequential manner. The changing body configurations that occur during the development of hopping can be described in a total-body (composite) sequence or in body-part (component) sequences. Limitations The following limitations existed in this study: 1. The subjects selected for this study were from an available population of children enrolled in summer motor programs at a community Y.M.C.A. and at a major university in the midwest. A random selection of subjects was made from this population for the development of a data-collection videotape. The observers for this study were limited to female, upper-division undergraduate students in either a kinesiology major or an 18 elementary education certification program at a university in the Rocky Mountain region. The observers for this study were limited to an available sample of female volunteers from either an undergraduate class in motor development or an undergraduate class in elementary physical education methodology. The observers for this study were assigned to observational groups based on scheduling availability. CHAPTER II REVIEW OF LITERATURE The purpose of this study was to examine the generalizability with which observers can classify the hopping performances of children according to changing body configurations described in composite and component developmental sequences. It is worthwhile to review the research literature from several perspectives: (a) the description of motor skill acquisition, (b) the issues of stage theory and developmental sequences, (c) the application of developmental theory to fundamental motor skills, (d) the relationship of observational skills to the analysis of developing movement patterns, (e) the use of generalizability analysis in determining the reliability of observational systems, and (f) the descriptive research that has been conducted on hopping. 19 20 Descriptive Research in Children's Motor Skill Achievements Interest in describing children's movement began flourishing during the second quarter of this century. Although much of the work during the 19203 and 19303 was conducted by psychologists and physicians interested in the behavior of infants (Bayley, 1935; Cunningham, 1927; Gesell. 1929; Jones, 1926: Shirley, 1931), physical educators were also directing their attentions to the movement of young people (Atkinson, 1924, 1925: Bliss, 1927: Wild, 1938). Most of this early work culminated in descriptive or normative accounts of movement during infancy, childhood, and adolescence. Describing overt movement in children has resulted generally in either product data or process data. Smoll (1979) describes these data as follows: Movement product data represent achievement of performance scores in specific motor skills, such as speed of running, distance of throwing, and height or distance of jumping. While this information is useful, the end result of a motor act tells little about its manner of execution. On the other hand, movement process data indicate how motor skills are performed, or more precisely, 21 they reflect the mechanical-actions of constituent motor patterns which are integrated in a space- time—force context. These data have been utilized to describe qualitative changes which take place as mature form is required in fundamental skills. (p. 21) In physical education, motor performance scores are generally considered to be movement product data. Motor scales, developmental charts, and most developmental assessment instruments are also examples of movement product data. Despite the developmental orientation of these various scales, charts, and assessment instruments, it is the movement process data that are most often associated with motor development. The objectives of process-oriented research in motor skill acquisition are to specify changing patterns of movement and to determine the variance and/or invariance of these patterns. The resulting data are best represented by verbal descriptions of phases in the course of development, or by specifically defined developmental sequences often referred to as stage sequences. Historically, qualitatively different behaviors have been identified as phases, levels, steps, and/or stages of development. Wickstrom (1983) noted that since the 1930s most behavioral changes in motor skills have been described 22 in practical, nontheoretical contexts and have been classified as developmental "stages". Recently motor developmentalists have questioned the terminology and the nontheoretical basis on which research has been conducted (Haywood, 1986: Roberton, 1977a). To interpret the differing research perspectives, an examination of the classical literature in motor development research is in order. 1 20 - 1940: The Foundgtion Years in the;§tudy of Motor Development The qualitative description of change has its roots in the early child development studies on infants. These early studies employed both direct observation and cinematography to examine reflexive development, the acquisition of upright locomotion, and prehensile abilities. Bayley (1935), Burnside (1927), Gesell (1929), McGraw (1932, 1943), and Shirley (1931) carefully analyzed the succession of behaviors leading to independent walking. Gesell (1929), H. M. Halverson (1932), and McGraw (1941b) were interested in the behavioral changes that occur in reaching and grasping. In addition, McGraw (1943) identified phases of development for the moro and grasp reflex, the swimming reflex, postural 23 adjustment, rolling, prone progression as depicted through creeping and crawling, and sitting. Upright locomotion was the focus of research conducted by Shirley (1931). She identified developmental movement sequences for creeping, for assuming erect posture, and for walking. Interested in whether these three sequences were part of a broader motor development scheme, Shirley used the mean age of task achievement to combine these sequences into a single chronology. Five groupings of stages were revealed: (a) passive postural control, (b) postural control of trunk E undirected activity, (c) active efforts at locomotion, (d) locomotion by creeping, and (e) postural control and coordination for walking. Although sequential shifting of stages occurred within a group, there was no transposition in the sequential order of the five groups. Shirley concluded that there was an orderly plan to the development of motor skills. Bayley (1935) was also interested in the question of invariable sequences. She tested Shirley's five groups for sequential ordering, but was not as successful in finding invariance. Reversals in the order of motor achievements did occur. Bayley concluded that it was not an ”inviolable (sic) sequence" of development that was impressive, but the progressive development of human functions. 24 Gesell (1929, 1939, 1946) and McGraw (1946) believed that the basis for progressive development was neuromuscular maturation. Advancing a strong biological orientation, Gesell maintained that maturation was due to "innate and endogenous factors". He did not believe, however, that these endogenous factors were invariant. In his principle of reciprocal interweaving he described development as a process of reincorporation and consolidation, but not one of hierarchical stratification. Gesell's 21—stages of flexor- extensor interweaving in the development of upright locomotion exemplified his beliefs on development. Like Gesell, McGraw (1932, 1941a, 1943) maintained that development involves smooth transitions, not the sudden emergence of behavior. She preferred to describe sequential development in terms of behavioral phases rather than stages. Whereas stages imply step-like development, phases are merely ”symbolic ratings" that represent the dominance of one configuration over another. McGraw believed that spurts, regressions, frustrations, and inhibitions of phases are part of the developmental continuum. The work by these early investigators substantiated the sequential emergence of motor ability during the first two years of life. Motor control was_found to develop in an orderly fashion and to follow a general direction. Bayley (1935), however, suggested that as the rate of development 25 decreases during the early childhood years, the variation in motor performance becomes more visible and motor functions become increasingly discreet and independent. By the late 1930s, child development research was ripe for studies investigating the motor development of young children. McCaskill and Wellman (1938: Wellman, 1937) were the first researchers to examine the sequential development of selected motor achievements in preschool children. Children 2 to 6 years were tested on a variety of activities involving steps and ladders, balls, jumping, and various locomotor activities. Each activity was differentiated further by the methods the children used to accomplish the task. Following data collection, these methods were ordered into seventy-six ”stages" according to the age at which 50 percent of the children were able to achieve success. McCaskill and Wellman (1938) found significant gains in the children's abilities from year to year. Wellman (1937) reported that monthly follow-up studies on 35 of the children revealed most children either maintained their same stage of motor performance or moved on to a higher level of performance. Regression to a lower stage was also found to occur in some children. Whereas McCaskill and Wellman used a laboratory environment to assess children's motor abilities, Gutteridge (1939) chose to study children in their natural habitat. To 26 obtain information about children engaged in activities of their own choice, Gutteridge trained teachers working with children ranging in age from 2 to 7 years to record and rate a variety of motor activities. The children's abilities were assessed according to a 14-point rating scale based on four discreet levels through which Gutteridge thought children passed as they mastered a motor activity: (a) no attempt, (b) habit in the process of formation, (c) basic movement achieved, and (d) skillful execution with variations in use. Gutteridge proposed that the slower rate of motor development in children after 3 years of age was due to a lack of environmental stimulation and challenge, rather than to the maturation of developmental patterns. She concluded that although variation in motor ability was apparent within and between children, evidence did exist for sequential patterns of behavior in the motor skill acquisition of young children. While Gutteridge, McCaskill, and Wellman were examining the motor abilities of young children from a general developmental perspective, Wild (1938) was interested in describing the development of a single fundamental motor skill. In examining the effect of children's neuromuscular development on the mechanical efficiency of their throwing pattern, Wild determined that children between the ages of 2 and 12 years display observable movement characteristics in 27 executing an overhand throw. Four clearly defined patterns of movement that suggested a developmental sequence emerged from Wild's work. Wild termed these patterns stages. She believed maturation to be the primary causative factor behind these developmental stages, but added that learning was also important as the more skilled patterns emerge from the basic patterns of movement. Wild's work on throwing is a classic in the physical education literature. It was the first work to examine the development of a specific fundamental motor skill and it has served as the prototype from which much of the current work on developmental sequences is based. It was not until the 1970s that work along these dimensions would continue. Between 1940 and 1970 interest in the qualitative aspects of developing movement waned. Research concerned with the quantitative measurement of children's movement, however, continued to flourish. 1940 - 1970: The Quantitative Description of Children's Motor Achievements During the 1940s, research in children's motor achievements turned from motor scales (Bayley, 1935: Cunningham, 1927: Jones, 1926: Shirley, 1931) and tests of motor proficiency (Atkinson, 1924: Bliss, 1927: Brace, 1927: 28 Carpenter, 1942: Cowan & Pratt, 1934: Johnson, 1932: McCloy, 1934) to the collection of achievement scores on a variety of gross motor and physical fitness items. Prior to 1940, Atkinson (1924, 1925) and Jenkins (1930) did study the motor achievements of adolescents and children 5, 6, and 7 years of age, respectively. Beginning with Espenschade's work (Espenschade, 1940, 1947), however, the descriptive research in motor development and motor skill acquisition began to focus more on the quantitative "performance” of fundamental motor skills and of sport skills (Curtis, 1975: Espenschade, 1940, 1947: Hanson, 1965: Johnson, 1962: Keogh, 1965: Latchaw, 1954: Vilchkovsky, 1972). Since 1940, studies have detailed the achievements of children on a variety of motor skills (Clarke & Petersen, 1961: Ellis, Carron, & Bailey, 1975: Glassow & Kruse, 1960: Kane & Meredith, 1952: Vincent, 1968), the relationship of growth and maturation to motor performance (Clarke & Petersen, 1961: Ellis, Carron, & Bailey, 1975: Espenschade, 1940: Malina & Rarick, 1973: Seils, 1951), the stability of performance across time (Clarke, 1971: Branta, Haubenstricker, & Seefeldt, 1984: Ellis, et al., 1975: Espenschade, 1940: Glassow & Kruse, 1960: Haubenstricker & Ewing, 1985: Rarick, 1973: Rarick & Smoll, 1967), and the changes in motor performance that occur as a result of 29 training or as a result of physical education programs (Dohrmann, 1964: Dusenberry, 1952: Earls, 1975). The measurement of motor performance during childhood and adolescence continues into the 19803 (Branta, Haubenstricker, & Seefeldt, 1984). Morris, Williams, Atwater, & Wilmore (1982) have recently extended these studies to early childhood by turning their attention to the motor performance of preschoolers. The research that has culminated in movement product (or motor performance) data has been abundant. Several excellent literature reviews expand on the results of the many studies that have been undertaken. Espenschade (1960) reviewed performance changes for running, jumping, throwing, and balancing. Post-1960 reviews include Branta et al. (1984), Espenschade and Eckert (1980), Eckert (1987), and Keogh and Sugden (1985). A more recent review by Haubenstricker and Seefeldt (1986) contains exceptional tables and graphs comparing pre-1960 and post- 1960 performance on a variety of skills, as well as tables listing the sources used in the preparation of the review. Although the measurement of quantitative performance has been and will continue to be useful in describing the amount of change that takes place among children as they develop, it fails to describe how that change is taking place. Between 1940 and 1970, a few studies examined the kinematic aspects of changing motor performances (Beck, 30 1965: Clouse, 1959: Dittmer, 1962: Halverson, 1958). Only three studies were conducted during this time period, however, that attempted to describe in qualitative terminology how children moved (Deach, 1951: Hellebrandt, Rarick, Glassow, & Carns, 1961: Sinclair, 1974). 1940 - 1970: TheAnglitgtive Deggription of Children's Motor Achievements In 1950, twenty-two years after Wild (1938) published her work on throwing, Deach (1951) examined the manipulative skills of children between 2 and 6 years of age. She defined three basic steps in motor pattern sequences for throwing, catching, kicking, striking, and bouncing. Deach admits that the scope of her study was limited by a cross- Isectional approach and by a small number of subjects in each age group. Still she was able to determine that patterns of performance increased in complexity in relation to stages of development rather than chronological age. Hellebrandt, Rarick, Glassow, and Carns (1961) considered the differentiation between maturational aspects <>f neuromuscular change and change due to learning as central to identifying and describing developmental and mature patterns of motor skills. They analyzed carefully the emergence of horizontal jumping, relating its 31 development in children to autogenous neuromuscular patterning. Interestingly, they determined that "jumping per se" is a phylogenetic ability that unfolds progressively, while the standing broad jump is an ontogenetic ability that is learned. Innate aspects of the broad jump are difficult to differentiate from learned aspects. The work of Hellebrandt et a1. suggests that maturation evolves into skilled voluntary movement. Although they contributed greatly to an understanding of the changing arm and leg action in jumping behavior, they did not propose a sequential model for describing these maturing actions. Culminating this period of limited motor development research was a study conducted by Sinclair (1974) in the late 19603. Sinclair expanded upon the work of Deach by employing a 3-year longitudinal research design in a study of 2 to 6 year old children. Congruent with previous research, she concluded that although behavioral variation is to be expected, identifiable sequences do occur in the motor development of children. Sinclair described both age level characteristics of motor ability and mature‘ characteristics of specific motor tasks, however she did not carder these characteristics along a developmental continuum from immature to mature behavior. 32 1970 - 1988: Currgnt Trends in Motor Develgppent Research The 1970s brought a new wave of interest to the study of motor development, renewing and expanding upon the work of Wild (1938) in the description of intraskill sequences. The concept of sequential motor development was not new, but the emphasis had shifted to providing teachers with information that would allow them to assess the development of their pupils by how they moved, rather than by how far or how fast or how high they moved (Halverson, 1971: Seefeldt et al. 1972). Today, process-oriented research has become the primary focus of descriptive research in motor development. Current research defines developmental sequences for fundamental motor skills in terminology that succinctly describes body actions during performance. It is the concern for helping teachers understand movement characteristics of children that has been the impetus behind this trend in motor development research. Preceding this renewed interest in the qualitative aspects of changing motor patterns, psychologists began examining sequential development from a stage theory perspective. This shift of attention in psychology away from a behaviorist perspective was partially the result of a growing interest in Piaget's work on cognitive development (Wohlwill, 1973). Piagetian theory, however, suggested 33 strict criteria for stage theories of development (Pinard & Laurendeau, 1969). In developmental psychology, as well as in motor development, these criteria have been a catalyst to research by those who advocate and those who oppose the concept of developmental stages. Prior to reviewing current views regarding motor development sequences, a discussion on stage theories of development is appropriate. Stage Theories of Development Although early psychologists interested in the motor development of children often referred to progressive motor skill achievements as stages of motor development, the classical interpretation of stage theory comes from the cognitive domain. Initially, American child psychologists were unreceptive to theories of cognitive development that were based on progressive stages of behavioral change (Wohlwill, 1973). Despite a maturational orientation, researchers in the motor domain also have been cautious in the acceptance of stage theory (Roberton, 1977a). Roberton (1977a) stated, "Probably the biggest problem with stages is that no one likes them" (p. 176). She proposed that stages imply stereotypic behavior, despite their attempts to describe ontogenetic development. Wohlwill (1973) noted that the lack of acceptance for stage 34 theories of development stems from three problems: (a) a failure to agree upon the meaning of stages, (b) the use of stages in an explanatory capacity rather than a descriptive capacity, and (c) a general mistrust of both the connotations of discontinuity and the maturational base implied by stage theories. Concurrence on the criteria for what constitutes developmental stages has not occurred. Brainerd (1978) noted that use of the word "stage" has occurred in three categorical forms. The first use has been in a metaphoric sense by which stages take on an aesthetic connotation, evoking images of behavioral traits. Brainerd used Erikson's (1982) theory of psychosocial development as an example, along with Kohlberg's (1963) stages of moral development. Undoubtedly Freud's (1961) psychosexual theory of development also would fit into this metaphorical category. Brainerd's second category in the use of stages for denoting behavior was description. Stage descriptions are arbitrary ones whereby development is divided along its continuum via any set of dimensions that seem realistic. Several models can be proposed for the same behavioral construct and they can all be valid. What distinguishes different descriptive models is their degree of abstractness. H. M. Halverson's (1932) model of infant 35 prehensile development reflects a clearly defined usage of descriptive stages, whereas Piaget's (1983) mental operations and cognitive structures are a more abstract representation of descriptive stages. The final category advanced by Brainerd gives stage theory an explanatory dimension. To qualify as an explanatory theory, the stages proposed must satisfy three criteria. First, and most obvious, they must be descriptive of some behavior that changes over time. Second, they must hypothesize antecedent variables that are responsible for the distinctiveness of the stages. Finally, the most difficult criterion to meet requires that the antecedent variables be measurable independently of the behavioral variables. Brainerd argued that it is the antecedent variables in the second and third criteria that qualify stages as explanatory theories rather than descriptive models. The antecedent variables explain behavior, and prevent circular reasoning that is based upon the behavioral variables. No current development theory exists that is illustrative of explanatory stages. Turiel (1969) viewed Kohlberg“s stages of moral development as explanatory concepts, as did Kohlberg (1963) himself. As previously noted, Brainerd considered Kohlberg's theory representative of metaphoric stages. Research conducted on Kohlberg's 36 developmental stages supports Brainerd's contentions (Holstein, 1976: Kurtines & Greif, 1974). Piaget (1983) perceived his stages of cognitive development as explanatory constructs. His rationale for their explanatory validity is based on his strict criteria for what constitutes a stage. Because Piaget's theory has dominated the qualitative developmental research (Wohlwill, 1973), it also has been in a position to receive the most criticism. Examining each Piagetian criterion, Brainerd (1978) presented a critical analysis in the use of Piaget's stages as explanatory constructs for cognitive development. Brainerd acknowledged that if the criteria could be shown to be sufficient in establishing the existence of stages, then Piaget's stages do represent natural or non-arbitrary groupings of behavioral traits. Consequently, determining the antecedent or causative variables would remain the empirical question. Piaget's Criterip'for Developmentgl Stages Pinard and Laurendeau (1969) have furnished the most complete explanation of Piaget's five criteria for developmental stages. The criteria are (a) hierarchization, (b) integration, (c) consolidation. (d) structuring, and (e) equilibration. Unless otherwise noted, the following 37 discussion of these criteria is based upon the work of Pinard and Laurendeau and on the penetrating analysis of the criteria by Brainerd (1978). Hierarchization. Hierarchization refers to a qualitative invariant, developmental sequence. The term sequence implies there is a fixed order to the appearance of qualitatively new behaviors. Invariant refers to the intransitivity of the developmental levels. Later stages do not precede former stages. Therefore the criterion of invariant sequence says stages exist if the "behaviors associated with them emerge in an order that cannot be altered by environmental factors" (Brainerd, 1978, p. 175). The implication is that maturational events are the antecedents to behavioral sequences, a belief promoted in the work of Gesell (1939) as he described the reciprocal interweaving of flexors and extensors in the attainment of upright locomotion. Brainerd distinguishes between maturational sequences that suggest unalterable genetic control, and measurement sequences that consist of culturally transmitted knowledge. Measurement sequences relate behaviors to one another in a fixed order based upon definitional connections postulated by the measurer. They may be, therefore, invariant sequences but they may or may not be based upon maturational events. Cultural definitions or environmental influences 38 cause these sequences to be arbitrary and subject to occasional fluctuation. Brainerd contends that use of the universal, invariant sequence criterion as evidence for a stage sequence is superficial. One would only need to show that behaviors of later stages precede those of former stages in order to identify a sequence as measurement rather than maturation. Flavell (1972) has pointed out that it is the discovery of sequence, whether measurement or maturational, that is an important first step in developmental inquiry. Integration qnd Congolidation. Integration of each preceding stage into the next immediate stage is the second criterion for the existence of stages. Integration entails both restructuring and coordination of behavior. According to Pinard and Laurendeau (1969), the restructuring of behavioral operations occurs in the transitional periods between stages. The coordination of new operations occurs at the next higher stage. Integration entails vertical delays (decalages) in the progression between stages, and is concerned with operations on homogeneous objects or problems. Consolidation, not to be confused with coordination, is the gluing together of an operation across different problems at the same developmental level. The consolidation process accounts for horizontal decalage, or delays of 39 development within a stage. Consolidation is the within stage complement to integration, and is concerned with operations on heterogeneous problems. Pinard and Laurendeau (1969) discuss integration and consolidation from the perspective of cognitive development and logical operations. A view of integration and consolidation with regard to motor development may further an understanding of these two concepts. The problem of object projection, for example, can be solved in three distinct ways: throwing, striking, or kicking. Generally, all three techniques proceed developmentally from ipsilateral patterns of movement to contralateral patterns of movement. Ipsilateral and contralateral control might be regarded as hierarchical levels, respectively, of motor development. Integration from an ipsilateral pattern to contralateral control could entail delays or fluctuations of performance if the object to be projected is changed, but the projection technique remains the same. These object changes might include the size, weight, or shape of the ball. In contrast to integration, consolidation of the contralateral pattern concerns the developmental relationship across throwing, kicking, and striking. Delays in contralateral consolidation would be indicated by developmental variation across the three projection .techniques, rather than across the objects projected. 4O Brainerd (1978) contends that integration and consolidation are_simply restatements of the sequence criterion. In other words, it is feasible that integration is simply a sequence defined by the size, shape, or weight of an object, and consolidation is a developmental sequence defined by the type of motor task. From Brainerd's empirical viewpoint, integration and consolidation are subject to the same constraints as the invariant sequence criterion. Specifically, are they maturation sequences and thus stages of development, or are they simply measurement sequences? §tructura1 Wholenegp. Perhaps the most abstract of Piaget's criteria, and at the same time an important characteristic of Piagetian stages, is the criterion of operational structures. Pinard and Laurendeau (1969) pointed out that the structural wholeness of a developmental stage would not be possible until complete mastery of that stage had occurred, until vertical (integrative) and horizontal (consolidative) delays were complete. Piaget (1983) further stipulated that the uniqueness of increasingly complicated structures lies in their non- additive composition, their integrative nature. Structures are therefore unique to the stages for which they are posited and, as such, should not be represented in the behaviors of different levels. It is this condition of 41 non-additive mental structures that distinguishes structural wholeness from simple consolidation of heterogeneous behaviors at the same level of development. To comprehend fully the formation of cognitive structures as described by Piaget (Brainerd, 1978: Piaget. 1983) requires an understanding of logic and abstract algebra. Essentially, Piaget's mental structures are abstractions of overt behavior. These mental structures explain behavior. Brainerd believed such theorizing to be circular in that the structures are nothing more than conceptual descriptions of behavioral manifestations. The functional relationship between Piaget's structures and behavior has never been established. The functional relationship between neuromuscular structures and motor behavior perhaps offers more promise with regard to identifying structural wholeness in movement. A more simplified description of structural wholeness for the movement specialist might reside in the image of a latticework of interconnected neuromuscular operations that allow an individual to perform all motor tasks with the same elements of control. Across-task performance would reflect an underlying mastery of a particular ability level, such as contralateral movement. According to Piagetian criteria. once contralateral neuromuscular structures become 42 established, then ipsilateral patterns should no longer be employed in movement. In motor development research, Roberton (1982) and Langendorfer (1987a) have attempted to test what they interpreted as structural wholeness by examining the horizontal delays in the development of throwing and striking. In view of Piaget's non-additive stipulations on operational structures, it may be that they have been examining the less stringent criterion of consolidation. Whereas consolidation is a prerequisite to structural wholeness, it does not infer strict adherence to the non- additive composition of evolving behavior. Operational structures would seem logically to exist as neural interconnections. Structural wholeness as conceived by Piaget, however, is much too strict a criterion for most developmental sequences (Flavell, 1985). Research in cognitive development has easily discredited Piaget's non- additive element of structural wholeness (Brainerd, 1978). Perhaps in the motor domain, as well as the cognitive domain, non-additivity is not a condition for structural wholeness. In the performance of motor skills, for example. a mechanically efficient performer may voluntarily move in immature, inefficient movement patterns. Eqpilibration. Equilibration is the final, and most recent, of Piaget's stage criteria, the one he considered 43 most indispensable (Brainerd, 1978: Piaget, 1983: Pinard & Laurendeau, 1969). Roberton (1978c) succinctly described the equilibration process as follows: An imbalance between the mental structures and the environment is supposed to cause a behavioral action system to move out of its consolidated stage into a transition between stages and, then, into the next higher stage as reorganizing structures consolidate again. Overt behavior will reflect this equilibration process by showing periods of relative stability when in a consolidated stage and periods of instability when changing to a higher stage (p. 65). It is in equilibration that Brainerd (1978) had the most optimism for a criterion by which to define explanatory stages of development, should they exist. Brainerd recognized that instability suggests possible correlations with antecedent or causative variables. Maturational events, such as the hormonal changes that produce the adolescent growth spurt, are illustrative of alternating phases of stable quiescence and behavioral instability. Research has shown that Piaget's stages, however, tend to emerge gradually (Flavell, 1985). Brainerd concluded that 44 evidence for this gradual emergence has caused many theorists to view development as a smooth, continuous process rather than a process of abrupt beginnings and endings. However, he further suggested that it could simply be a case of large scale continuity masking discontinuity and that discontinuity may occur at the level of individual concepts. The research that has been conducted to analyze Piagetian stages of cognitive development has caused many developmentalists to deny the explanatory thesis of stage theories. They concur with Brainerd (1978) that these stages are simply arbitrary descriptors of development. That evolving behavior can be described by qualitatively different periods in the life span is well documented in both the psychological development literature and the motor development literature (Erikson, 1982: Flavell, 1985: Gesell, 1946: Holstein, 1976: Kohlberg, 1963: Roberton. 1978: Seefeldt, 1972a: Wohlwill, 1973). By the same token, few developmentalists would deny that there are trends in the sequence of events between birth and adulthood. Since the existence of stages as an explanatory construct for development has been so difficult to confirm, alternative approaches to the qualitative study of developmental trends have been offered. 45 Developmental Sequences vs. Developmentgl Stages Research that examines sequential development from the perspective of qualitative change is concerned with two problems: (a) determining the amount of behavioral invariance that occurs over a population, and (b) determining the interpatterning among sets of qualitatively different behaviors (Wohlwill, 1973). The first issue refers to the robustness of a developmental sequence. It is the second issue, the interpatterning of behaviors, that is purported by Flavell (1985) and Wohlwill (1973) to be concerned with stages. Apparently the description of qualitative behavioral change may assume one of two forms, that of developmental sequences and that of developmental stages. In distinguishing between stages and sequences, Flavell (1985) noted that both are descriptions of development denoting relationships between behaviors over time. According to Flavell, what identifies each is the temporal appearance of the behavioral components. Whereas developmental sequences describe systematic asynchrony of temporally ordered events, the behavioral components in a stage sequence develop in a synchronous or step-by-step fashion. Wohlwill (1973) clarified this distinction by classifying the components in a stage sequence as "nodal D. 46 interrelationships among two or more qualitatively defined variables developing apace" (p. 192). In other words, stage sequences are defined by two or more developmental sequences with the same behavioral features during their developmental course. Wohlwill (1973) has advanced a view of developmental stage sequences that encompasses both Flavell's step-by-step concept of the synchronous development between sequences and Piaget's concept of horizontal decalage. This prototype for stage theory offers four models of developing interrelationships: (a) synchronous, (b) horizontal decalage, (c) reciprocal interaction, and (d) disequilibration-stabilization. Wohlwill suggests that if, in fact, stages of development do exist, they represent interdependencies among different tasks or concepts in the process of development. Flavell (1972, 1985) has chosen to emphasize sequences, rather than stages, in describing qualitative changes in cognition. Brainerd's (1978) analysis of hierarchization and integration made it clear that hypothesized developmental sequences can be classified as either maturational sequences or measurement sequences. Flavell (1972) augmented this distinction by suggesting that developmental sequences can actually be explained from three perspectives: (a) the structure of the organism, (b) the 47 structure of the environment, and (c) the structure of the items. In addition, the sequential acquisition of behaviors according to these three structures is not limited to integration. Components of a developmental sequence may be hierarchically related to each other through addition, substitution, modification, inclusion, or mediation. The interpretations of Flavell (1972, 1985) and Wohlwill (1973) have given some flexibility to the Piagetian criteria for stage theories and yet offer some structure for a developmental theory based on qualitative change. The semantic confusion between stages and sequences still exists, clouding the important question of just how robust sequence descriptions are. This is readily apparent in the current motor development literature (Haywood, 1986: Ulrich, 1987). Much of this confusion is a reflection of the maturational arguments associated with a stage concept, along with the use of the term stage to describe temporally sequenced events (Wohlwill, 1973,). The early work in motor development is indicative of this purely descriptive usage (Bayley, 1935: Gesell, 1939: H. M. Halverson, 1932: Shirley, 1931), where very little emphasis was placed on developmental ”stages" as theoretical constructs in the classical Piagetian tradition. The work currently being conducted on qualitative changes in fundamental motor skills follows two different 48 historical perspectives. The research at Michigan State University reflects the descriptive use of stage sequences as they were employed in the classic motor development research (Haubenstricker & Seefeldt, 1986). The paradigm followed at the University of Wisconsin has attempted to relate classical Piagetian theory of stages to the study of motor skill acquisition (Roberton, 1977a). Current Application of Developmental Sequences To Fundamental Motor Skills In the late 19503 and early 19603 there was a renewed interest in the process of movement. Product scores had, provided the physical educator with trends in the performance of children across age, but these scores alone were not sufficient to assess changing movement patterns. Mechanical efficiency was beginning to play an important role in the analysis of mature movement (Broer, 1960, 1966, 1973: Godfrey & Kephart, 1969). The application of mechanics to the analysis of developing motor patterns had also begun to take place (Halverson, 1958: Clause, 1959: Dittmer, 1962: Fortney, 1983: Beck, 1965: Vilchkovsky, 1972). What was needed by the physical education teacher, however, was a method for recognizing, describing, and understanding the overt body configurations through which 49 children progressed toward mature performance (Halverson, Roberton, & Harper, 1973: Halverson, 1971: Seefeldt et al., 1972). In 1962, at the University of Wisconsin, Lolas Halverson (Halverson et al., 1973) began a small, longitudinal study on the developmental motor patterns of children. Through the use of cinematography she began identifying the temporal and spatial displacements of body parts during children's performance of fundamental skills. Working with Halverson, Mary Ann Roberton has been eminent in encouraging the heuristic value in the study of motor stages (Roberton, 1977a, 1978c) In 1966, at Michigan State University, Seefeldt and colleagues, (Branta et al., 1984: Seefeldt & Haubenstricker, 1982) undertook an extensive longitudinal study on physical and motor development. In conjunction with this study they began viewing film of children's movement from two perspectives, the order of appearance of movement patterns and the mechanical proficiency of those patterns. These two groups of researchers and their students have been tremendously influential in providing the physical education teacher with developmental sequences for several fundamental motor skills (Clark & Phillips, 1985: Halverson & Williams, 1985: Haubenstricker, Henn, & Seefeldt, 1975: Haubenstricker & Seefeldt, 1976: Langendorfer, 1987a, 1987b: Poe, 1970: 50 Roberton & Halverson, 1984: Sapp, 1980: Seefeldt, 1972b: Seefeldt & Haubenstricker, 1974, 1976a, 1976b, 1976c: Seefeldt, Reuschlein, & Vogel, 1972: Williams, 1980). Research on fundamental motor skill sequences has not been limited to the work conducted at the University of Wisconsin and at Michigan State University. McClenaghan and Gallahue (1978), Gallahue (1982), and Wickstrom (1983) also have been influential in describing the developing motor patterns of children. Haubenstricker and Seefeldt's (1986) recent review, Acquisition of Motor Skills During Childhood, furnishes an excellent list of individuals who have hypothesized developmental sequences for thirteen different fundamental motor skills. A comparison of the work conducted by Roberton and Halverson with that conducted by Seefeldt and Haubenstricker stands out as being most interesting, however, because of the differences in the philosophical framework from which their sequences are hypothesized. Criteria for Developmental Motor Sequences In the nontheoretical sense, motor stages are movement patterns characterized by qualitatively distinct spatiotemporal organizations that present themselves on progressive levels as an individual moves toward mature 51 motor performance (Roberton, 1977a: Roberton & Langendorfer, 1980: Seefeldt 1980: Wickstrom, 1983). Explaining how they identified stages of development, Seefeldt et al. (1972) indicated that a stage needs to demonstrate "sufficient commonality" among children as they perform a skill, and it must be perceptible by experienced observers. Moreover, Seefeldt and Haubenstricker (1982, pp. 311-312) indicated that each new stage should describe a more mechanically efficient performance occurring as a result of one or more of the following: 1. a greater range of movement around the force- producing points: 2. the addition of more joints to the ‘power train': 3. a more continuous, or less interruptive, flow of the movement: 4. better positioning of the body for maximum force production. The work on fundamental motor skills at Michigan State University is based on the well-researched premise that human movement develops in an orderly and predictable fashion, under maturational control but subject to environmental intervention (Branta, et al., 1984: Seefeldt, et al., 1972). To Seefeldt and his colleagues, a series of stages represents guideposts to development. Their stages are defined in terms of arbitrary, ordinally-scaled points 52 not unlike those of Gesell (1939), H. M. Halverson (1932). McGraw (1943), Shirley (1931) and, of course, Wild (1938). Although the series suggests an implicit order, invariance is not a criterion for the Michigan State stage sequences. Branta, Haubenstricker, and Seefeldt (1984) do not see omissions and reversals as invalidating a proposed sequence. They view development as continuous, rather than "step-like" (Haubenstricker & Seefeldt, 1986), but their use of terminology often confounds their philosophical position. This is evidenced in two aspects of their work. The first is their use of the term "stage" in an atheoretical sense as opposed to the theoretical implications of stage-theory in the developmental psychology literature. The second is their attempts to describe transitions between stages. Whereas they have characterized shifts between stages by ”abrupt changes in the positioning of one or more limbs or body segments" (Seefeldt & Haubenstricker, 1982, p. 311). they have also stated that "progression from stage to stage does not imply abrupt change" (Branta, et al., 1984, p. 407). Although earlier works by these researchers presented some discongruities in terminology, their current views on developmental sequencing are more clearly delineated (Branta, et al., 1984: Haubenstricker & Seefeldt, 1986). The concern of Seefeldt and his colleagues (Seefeldt et al.. 1972) has been to supply the elementary school physical 53 education teacher with developmental progressions in fundamental motor skills that can be used to refine curricula and to assess the motor status of students. Researchers at the University of Wisconsin also have been interested in the applied aspects of developmental sequences (Halverson, 1971: Halverson et al., 1973: Roberton, 1977a: Roberton 1987). From the beginning, their collection of data has included biomechanical information and has focused on the changes in body configuration as a skill develops over time (Halverson, et al., 1973). In 1977, however, Roberton added a new dimension to the study of motor development. In an attempt to approach the description of motor skill acquisition within a theoretical framework, she proposed that motor stage research be guided by the theoretical criteria established in the Piagetian literature (Roberton, 1977a). Piaget (1960) had indicated that the structure of an hypothesized stage theory might only encompass one or two criteria from his original five. Roberton (1977b, 1978c, 1982) chose the concepts of intransitivity and universality as the most testable aspects of a motor stage theory. Intransitivity implies stability of performance within a stage and adjacency of performance during transition between stages. Therefore, to test for intransitivity several trials or performances must be examined within time. 54 Universality, in contrast, can only be proven through longitudinal testing (Roberton, 1977b). Roberton (1977b, 1978b) proposed a within time/across time paradigm to undertake the testing of hypothesized motor stages in the development of throwing. She also established strict levels of acceptance for what constituted stability of performance for motor stages. If a child shows 50% consistency in performance across trials, then a child is said to be stable for a stage. Variation in performance might indicate transition between stages, and should therefore only occur to adjacent stages. A stage sequence would be refuted if variation in performance occurs to non-adjacent stages. Although Roberton (1978a) has presented partial longitudinal evidence in support of her proposed stages, most of her longitudinal work (Roberton & Langendorfer, 1980) has been centered on a small number of subjects. She has, however, discreetly followed a plan of research using stage theory as a heuristic model for her work (Roberton. 1982: Roberton, Williams, & Langendorfer, 1980). When her results did not provide strict support for universality and intransitivity, Roberton (1978a, 1978c) suggested new ways of viewing theoretical terminology and new models by which to conduct research. Guided by Wohlwill's (1973) claim that stages entail nodal interrelationships, Roberton (1978c) recommended that 55 the levels within skill sequences be called steps rather than stages. Based on the theoretical model of developmental stages employed in psychology (Flavell, 1985: Wohlwill, 1973), Roberton's recommendation holds merit. Since motor development research previously had not been conducted under this paradigm, however, her recommendation has resulted in semantic confusion. Haywood (1986) states: ”Developmentalists have yet to adopt a standard definition and usage of the ‘stage' concept: hence, the concept has been used in different ways....The terms ‘sequences,’ ‘patterns,’ ‘steps,’ and ‘phases', all might be used....Students of motor development, however, should expect that these terms and the term stage will sometimes be interchanged in the literature" (pp. 16-17). Acting upon Roberton's (1982) recommendation that motor stages should be examined across tasks, Langendorfer (1987a) has begun research into the nodal interrelationships of developing motor skills. He suggests that the integration of Piagetian stage theory with motor stage theory is important to an interactional perspective of development. At the same time, Langendorfer (1987b) has proposed that two sets of criteria are necessary in studying the sequential development of motor skills. He offers criteria for both general developmental sequences and the more robust stage sequences. Although it does not resolve the semantics 56 issue, Langendorfer's recent contribution may prove to be beneficial in differentiating between the applied and the theoretical work being conducted in motor development research. Conceptualizing Developmental Motor Sequences The development of motor skills occurs in three different ways: (a) between skills, (b) within skills, and (c) across skills. Development between skills is denoted by interskill or intertask sequences. An interskill sequence involves a chronology of tasks than can be ordered according to their age of acquisition, such as the steps that lead to upright locomotion proposed by Shirley (1931), or according to the level of difficulty, such as the teaching progression used in Red Cross swimming instruction. Seefeldt et al. (1972) suggested that curriculum in physical education could be guided by interskill teaching sequences. Intraskill sequences represent the changes in body configuration that occur as an individual progresses from immature performance to mature performance in a single skill.' H. M. Halverson (1932) and Wild (1938) proposed the first intraskill sequences for prehension and throwing, respectively. Intraskill development is currently the focus of most motor skill research. The third category of motor 57 skill acquisition, across skills development, refers to the relationship of intraskill sequences to one another. According to Flavell (1985) and Wohlwill (1973), this is the relationship that would support the "horizontal structure" of developing motor abilities and a developmental stage theory in motor skill acquisition (Langendorfer, 1987a: Roberton, 1982). Intraskill sequences can be further differentiated by whether they describe changes in the configuration of the total body or changes in the configuration of body parts. Total body configurations, hereafter referred to as composite sequences, have been the basis for most of the proposed fundamental motor skill sequences. Wild (1938) used a composite approach when she identified her four stages of throwing. The researchers at Michigan State University have chosen to use a composite approach in designating developmental sequences for ten different fundamental motor skills (Haubenstricker, Henn, & Seefeldt, 1975: Haubenstricker & Seefeldt, 1976: Sapp, 1980: Seefeldt, 1972: Seefeldt, E Haubenstricker, 1974, 1976a, 1976b, 1976: Seefeldt, Reuschlein, & Vogel, 1972). Branta et al. (1984) indicate that while they do not believe that the action of all body parts develops in a lockstep fashion, there is enough cohesion between body parts that a total body configuration can be used to describe developing movement. 58 The number of steps in the developmental sequences described by Seefeldt and his colleagues varies depending upon the complexity of the skill being described.' Throwing, for example, is a rather complex skill and has been identified as having five steps to mature performance. The development of a mature pattern in throwing takes a number of years (Early Childhood Motor Skills Development Study (Early Childhood), 1985: Roberton, 1978b: Roberton & Langendorfer, 1980: Seefeldt & Haubenstricker, 1982). In contrast, skipping develops later than throwing but a mature pattern is reached in skipping in a much shorter period of time (Early Childhood, 1985: Seefeldt & Haubenstricker, 1982). Only three stages (Haubenstricker & Seefeldt, 1976) have been hypothesized in the development of skipping, of which the last two might simply be progressive refinements in form rather than distinct changes in body configuration (Wickstrom, 1987). As Wickstrom (1983) noted: ”The number of stages inevitably varies from skill to skill and from one investigator to another according to available data and intended use. There is an arbitrary factor in the process" (p. 13).. Gallahue (1982) acknowledged the arbitrary factor in the 3-stage approach outlined by himself and McClenaghan (1978). He indicated that while a more complex sequence may be appropriate to a sophisticated observer, a three stage 59 approach serves as an easy method by which the physical education teacher can assess motor development in students. A number of locomotor, stability, and manipulative skills have been sequenced by Gallahue (1982) and Gallahue and McClenaghan (1978) using the following guidelines: Initial Stage: Characterized by the child's first observable attempts at the movement pattern. Many of the components of a refined pattern, such as the preparatory action, and follow-through are missing. Elementgry Stggg: A transitional stage in the child's movement development. Coordination and performance improve, and the child gains more control over his movements. More components of the mature pattern are integrated into the movement, although they are performed incorrectly. Mature Stage: The integration of all the component movements into a well-coordinated, purposeful act. The movement resembles the motor pattern of a skilled adult (in terms of control and quality, but it is lacking in terms of movement performance as measured quantitatively) (Gallahue, 1982, p. 179). 60 Roberton (1977a)_introduced a component part approach to the study of developing motor skills. The component approach defines sequential changes in the configuration of body parts such as the movements of the arms separate from the movements of the trunk. By examining the developmental steps of the various body segments, total-body profiles of the developing skill can be obtained. Roberton (1977b, (1982) has shown that these profiles are not the same for all children, which signifies that the development of component parts may proceed at different rates in the same individual or in different individuals. The component approach of describing developmental motor sequences is better suited than a composite approach to the theoretical stage constructs on which Roberton has based her research. Whereas some of her component sequences have been shown to be universal and intransitive (Roberton, 1977b, 1978a, 1982), the variability found in her total-body profiles does not support universal and intransitive stages. Roberton (1977a) has suggested that composite sequences emphasize one component of configuration more than others. She has also proposed that observers of movement tend to ‘watch parts of the movement rather than the whole movement. Seefeldt (personal communication, December 14, 1987) contends that segmental movements develop in relation to a critical component and that these relationships are 61 sufficient enough to describe cohesive, composite configurations. Seefeldt's perspective suggests that although there may be variation in the profiles of body components, this variation might be limited to a discreet range of possibilities. Roberton's profiles (1982) have indeed suggested "biomechanical constraints", "common linkages", and "diversified sequences of change within a non-random range of profiles". How extensive the range of profiles is has yet to be determined. Perhaps the difference between composite sequences and component sequences is simply the preciseness with which the developing movement is observed. Regardless of the observational method or the philosophical belief, there are principles upon which motor developmentalists agree. Foremost, it is well accepted that there is developmental progression in the acquisition of fundamental motor skills (Branta et al., 1984). It is also recognized that longitudinal studies are imperative in the validation of developmental sequences (Wohlwill, 1973: Roberton, Williams, & Langendorfer, 1980: Seefeldt, 1972a). Longitudinal research involves tremendous expenditures of time, money, and energy. While it cannot be replaced as an important validator of changes across time, research methodologies have been advanced that screen for the potential requisition of longitudinal studies (Roberton & 62 Langendorfer, 1980: Roberton, Williams, & Langendorfer, 1980). Prelongitudinalvgcreening;Methogs for the Stqu of Developmental Motor Sequences Two prelongitudinal screening paradigms have been employed to pilot test developmental motor sequences for subsequent longitudinal validation: (a) within-time/across- trial testing has been proposed to screen hypothesized stage sequences, and (b) frequency distributions of stages across age has been used to screen hypothesized developmental sequences. Both paradigms can be used with either cross- sectional data or mixed-longitudinal data. The prelongitudinal screening for stage sequences entails testing the criteria of intransitivity and universality (Roberton, 1978b). As previously discussed in this review, Roberton (1977b, 1978b) proposed a within- time/across-time paradigm to test the hypothesized motor stages in the development of throwing. The within-time aspect of this paradigm involves idiographic trial-to-trial testing for stability and adjacency of performance. Roberton has found support for humerus and forearm stage sequences in the development of the overarm throw using this 63 paradigm (Roberton, 1977b, 1978a, 1978b: Roberton & Langendorfer, 1980). In order to pilot test for a developmental sequence, Roberton, et al. (1980) proposed a population model that illustrates the frequency with which the stages or levels in a sequence occur. The frequencies of each stage or level for any given cross-sectional sample can be graphed across age. The shape and relationship of the curves should depict the lower developmental levels declining across age as the higher levels increase in frequency. If a developmental sequence has been hypothesized correctly, the prelongitudinal frequency graph would resemble the proposed population model of a longitudinal sample. Prelongitudinal screening has been used successfully in hypothesizing developmental sequences fora number of fundamental motor skills (Clark & Phillips, 1985: Halverson & Williams, 1985: Haubenstricker et al., 1989: Haubenstricker, Branta, & Seefeldt, 1983: Haubenstricker, Seefeldt, Fountain, & Sapp, 1981: Langendorfer, 1987a: Williams, 1980). Langendorfer (1987b) was the first to employ both prelongitudinal paradigms in a study of eight different body-component sequences in overarm striking. He consolidated the research literature on the sequential development of fundamental motor skills by proposing two distinct sets of criteria, one for developmental sequences 64 and one for stage sequences. The criteria for develOpmental sequences are as follows: 1. Comprehensiveness -- are only hypothesized movement levels observed for all subjects? 2. Sequence Order -- are the sequence levels in the same order as in the developmental model? 3. Sign of the function -- do the frequencies of levels change as hypothesized in the longitudinal model? 4. Level modality -- do each of the levels demonstrate a modal occurrence at some place in the sample age range? The criteria for stage sequences include the four items above as well as the following: 1. Inclusiveness -- are all hypothesized movement levels observed within the total sample? 2. Stability -- do all subjects demonstrate one hypothesized level a majority (60*) of trials? 3. Adjacency -- do all subjects demonstrate across-trial adjacency of levels? In using both sets of criteria to screen overarm striking, Langendorfer concluded that the sequence criterion of level modality should undergo further investigation 65 before it is recommended for prelongitudinal screening of developmental sequences. This recommendation is supported in the composite stages for throwing proposed by Haubenstricker, et al. (1983). The remaining criteria, however, present a conceptual model for differentiating between developmental sequences and stage sequences. Researchers are left to choose those criteria that best fit their philosophy. According to Langendorfer, the more conservative researcher may choose the more robust stage criteria for testing hypothesized sequences. Although Langendorfer's proposal has the potential for alleviating several of the discrepancies in defining and describing developing motor patterns, the work has only begun in determining the usefulness of these descriptions to the practitioner. Herkowitz (1978) proposed that developmental task analysis be used to analyze the effects of environmental factors on motor skill performance. Roberton (1987) and Langendorfer (1987b) have recently examined some of the effects of environmental constraints on the developmental levels of children throwing and striking. An understanding of environmental constraints and developmental task analysis holds great promise as a teaching aid by which to construct developmentally appropriate learning environments. It is of little value, 66 however, if teachers are unable to determine the performance levels of children. The relationship between valid developmental sequences and their usefulness to the clinician needs to be determined (Roberton, 1989). Although continued research is necessary in both describing and validating developmental sequences, prelongitudinal validation of several sequences has been conducted (Clark & Phillips, 1985: Halverson & Williams, 1985: Haubenstricker et al., 1989: Haubenstricker et al., 1983: Haubenstricker et al., 1981: Langendorfer, 1987a: Roberton, Williams, & Langendorfer, 1980: Williams, 1980). Concurrent with continued efforts to determine their longitudinal validity, research examining the usefulness of these sequences to teachers or therapists can now take place. The first question to be asked with regard to usefulness concerns the ability of practitioners to identify developmental levels accurately and reliably. Clearly the ability to analyze a child's performance is important to challenging that child with appropriate tasks, as well as in determining whether program goals have been met. 67 Observational Skills As Related to Motor Development Observing and interpreting the motor behavior of children form the basis for developing teaching strategies aimed at enhancing individual improvement in motor ability. Through both formal and informal assessment of children's abilities, teachers are able to determine the success of their intervention techniques (Roberton, 1989). The implications of developmental motor sequences for teachers in physical education classrooms are many. Certainly the knowledge that teachers possess about sequential development of motor skills, along with their skill in observing developing movement patterns, should assist them in providing appropriate experiences for children (Roberton & Halverson, 1984: Seefeldt, 1980). Coincident with current research interests in developmental motor patterns is a growing interest in the skill of observation (Allison, 1987: Barrett, 1977, 1979b: Bell, Barrett, & Allison, 1985: Barrett, Allison, & Bell, 1987: Cheffers, 1977: Fishman & Anderson, 1971: Hoffman, 1974, 1977a, 1977b, 1983: Locke, 1972: Petrakis, 1986, 1987: Radford, 1988). Hoffman (1974) has stressed that skill analysis in the gymnasium should be qualitative not quantitative, that it should be "utilitarian rather than acad educ beha motc (Ba: res Tat See 9t 31: 5i: fr: 3h. 9X 68 academic". Observation of motor skill by the physical education teacher is conducted in order to assess both motor behavior and the instructional practices used to change motor behavior. Hoffman (1974, 1977a, 1983) and others (Barrett, 1979b: Locke, 1972: Radford, 1988) have expressed a concern about teacher preparation programs that emphasize skill analysis as a process by which to understand the mechanics of performance, and all too often overlook the importance of skill analysis to pedagogical concerns. Motor development specialists have long expressed a belief that developmental sequences not only service researchers as heuristic tools, but they are useful as rating scales or checklists for teachers (Roberton, 1977a: Seefeldt et al., 1972). Used in such a capacity, Seefeldt et al. (1972) have suggested that the identified levels along a developmental continuum must be easily observable. Similarly, Roberton (1977a) stressed that checklists formed from developmental sequences must be useful to teachers. She cautioned that validating developmental sequences and examining the usefulness of these same sequences are separate research issues. Addressing the topic of future directions in the applied aspects of motor development, Roberton (1989) has called for field research that tests the ability of practitioners to identify developmental patterns of movement accurately and reliably. 69 Only recently have a few studies been conducted that are concerned with the usefulness of developmental sequences (Mosher & Schutz, 1983: Ulrich et al., 1988). Allison (1987) noted that most of the research on the ability to observe and analyze motor performance has centered on the relationship between either (a) skill experiences and observational ability or (b) observational experiences and observational ability. It is worthwhile, therefore, to review the literature on the ability of individuals to observe and rate motor skills. Observation in the Instructional Process No one seems to deny the importance of observation to the interpretation of performance and the subsequent development of instructional strategies and tactics. Strasser (1967) proposed a conceptual model of instruction consisting of four interactional aspects in the teaching process: (a) teacher planning: (b) initiatory teacher behavior: (c) teacher observation, interpretation, and diagnosis of learner behavior: and (d) teacher behavior as influenced and influencing. The third aspect of his model was a tactical element loop involving observation, interpretation, diagnosis, and behavior. Hoffman (1974) presented a three-level model very similar to Strasser's 7O tactical loop that again emphasized the importance of observation to interpretation and diagnosis. Arend and Higgins (1976) discussed the process of observational analysis as a three-phase strategy for the systematic analysis of movement. Pre-observation is the planning phase that entails the classification and description of movement, thereby establishing an expectancy set for the observer. Observation, the second phase of movement analysis, culminates in detailed records of the motor performance. According to Arend and Higgins, a performance can be recorded from direct-subjective viewing of the performance, cinematographic or videotape techniques for later viewing, and/or electromyographic techniques to obtain muscle responses. They suggest that direct— subjective observations collected systematically over repeated trials are the most useful for the teacher. Cinematographic or videotape techniques, however, are useful for direct-systematic analyses. Finally, the third phase of the systematic analysis of movement is the post-observation phase, or the time during which the performance is evaluated for its efficiency. A strong argument for systematic observation was made by Cheffers (1977), who viewed the process of observation as being either inductive or deductive. Inductive observation often results in the development of deductive observational 71 methods and forms. Cheffers identified seven categories of deductive methods, one of which was product systems. Used as a method of quantifying the quality of performance, a rating scale would be considered a product system by Cheffers. Thegggill of 0p§ervgtion Although the importance of observation to the analysis of movement had been recognized, observation as a skill had not been studied extensively prior to the mid 1970's (Barrett, 1977). In 1972, Locke challenged the presumption that students preparing for a career in physical education would develop a generic ability to observe and analyze movement through their coursework in kinesiology or biomechanics. Subsequently, Barrett (1979b) emphasized the need for teaching the skill of observation as distinctly different from teaching the analysis of movement. In a series of studies, Barrett and others (Bell, Barrett, 8 Allison, 1985: Barrett, Allison, & Bell, 1987: Allison, 1987) examined the focus of preservice physical education majors observing an instructional setting for elementary children. They found that when physical education majors beginning programs of study in teacher preparation observed and recorded their reactions to an 72 elementary physical education lesson, only 10% of their statements focused on the movement responses of the children. Their remaining statements were directed toward the social and personal characteristics of the children, the personal characteristics of the teacher, and the teaching techniques used by the teacher. By the senior year of preservice training, however, the majors increased the number of statements they made about children's movement responses to 66.1%. Training in observational techniques appears to be paramount in the ability to observe movement. Preobservational planning is an important aspect of this training and, as noted by Arend & Higgins (1976), may take a variety of forms. One of the responsibilities of researchers is to produce observational instruments that reflect this variety of perspectives by which observation may take place. Even though preobservational planning focuses on a specific perspective, the observational ability of individuals will undoubtedly vary depending upon personal biases and expectancy sets. The next section provides a review of the research literature that has focused on the experiences and the background the observer brings to the observational setting. 73 The Relgtionship of Kinesthetic Experiences and Teaghing EMperiences to stervational Skill One area of study in skill analysis has been the relationship between the observer's motor skill ability and the observer's analytical ability (Allison, 1987: Armstrong, 1977). Armstrong identified several physical education texts that place importance on skill learning for the enhancement of observational abilities. Interestingly, the scant research that has been conducted on this topic has been inconclusive. Several studies have examined the association between the ability to analyze a performance of a motor skill and the ability to perform that skill. Girardinand Hanson (1967) studied the relationship between the ability to perform tumbling skills and the ability to assess the performance of tumbling skills. They found a significant correlation of .49 between performance ability and diagnostic ability, as well as a significant correlation of .51 between knowledge about tumbling skills and diagnostic ability. Conversely, Osborne and Gordon (1972) found that an observer's skill in a forehand tennis stroke was not a factor in the ability to rate accurately a model performing the same movement. 74 Both the Girardin and Hanson (1967) study and the Gordon and Osborne (1972) study were concerned with specific sport skills. Armstrong (1977) chose to use a novel movement task in testing the relationship between kinesthetic experience and analytic ability. He provided three groups of subjects with varying amounts of kinesthetic experience in the performance of a novel task and then trained all subjects in the identification of elements critical to performance in the same task. When the subjects were asked to analyze twenty filmed variations of the model task, the extent of their movement experiences was found to be unrelated to their analytic ability. A later study by Armstrong and Hoffman (1979) examined the relationship of teaching experience, rather than kinesthetic experience, to analytic ability. They did find a difference in the ability of experienced and inexperienced tennis teachers to identify performance errors in a tennis forehand. They suggested that this difference was due to experience in teaching, not in playing. 3 e Relationship of Training Specificity to opservational I kill 02 A second area of study on observational skill analysis has been concerned with differences in analytic ability 75 between physical education specialists and nonspecialists. Biscan and Hoffman (1976) compared experienced physical education teachers, undergraduate physical education students, and junior high school classroom teachers in their ability to indicate whether a motor skill performance matched a specific prototype. The physical education teachers and students performed better than the classroom teachers in the analysis of a cartwheel, a skill with which most individuals are familiar. There were no differences, however, between the three groups in the analysis of a novel skill. Biscan and Hoffman concluded that professional training in physical education enhances teachers' ability to form "criterion images" about specific sport performance, but does not result in a generic ability to analyze movement. Petrakis (1986, 1987) has approached the novice/expert research by analyzing the visual search patterns and ocular fixations of observers viewing live motor performances. These visual search patterns provide information about the search strategy used to focus on critical elements in movements. In one study (1986), Petrakis examined the number and the duration of eye fixations and visual scanning patterns for novice and expert tennis teachers viewing five forehand drives and six serves of a tennis player. In a second study, she examined the visual search patterns of 76 novice and expert dance teachers viewing two different dance performances. 'The results indicated that there are differences in the search patterns of novice and expert observers. Morrison and Reeve (1986) have suggested that because elementary school physical education is often under the direction of classroom teachers, it is important to determine their ability in motor skill analysis. Following a study in 1984 in which Morrison, Reeve, and Harrison determined that elementary education majors could effectively analyze motor skill after viewing instructional videotapes, Morrison and Reeve (1986) conducted a study in which they trained elementary education majors enrolled in an undergraduate physical education course to analyze specific physical skills for correct and incorrect movements. They then tested their subjects' abilities to apply this knowledge in the analysis of non-related skills. The results supported specificity of training in the analysis of motor skill and corroborated the findings of Biscan and Hoffman (1976). Fishman and Anderson (1971) contend that an important aspect of research in teaching is to select a significant perspective from which to evaluate children's performances. While such a perspective is often a reflection of a philosophical, theoretical, or personal bias, it can also 77 insure more complete descriptive records. With the exception of Mosher and Schutz (1983), and Ulrich et al. (1988), all of the studies on observational skills presented thus far have examined movement either from the perspective of what constitutes correct and incorrect movement or from the perspective of visual scanning patterns. These perspectives have been applicable to the analysis of skilled performance, but have yet to examine observational skills from a developmental viewpoint. The very nature of developing movement implies that one cannot judge a performance as right or wrong, correct or incorrect. A developmental perspective requires the observer to interpret movement along a developmental continuum. Choosing a developmental perspective, Robinson (1974) compared the competency of physical educators and elementary educators in classifying into "stages" the motor performance of children executing throwing, catching, and running patterns. While elementary educators tended to underrate performance in all three gross motor skills, significant differences were not found between the two experimental groups in their ability to rate children's performance along a prescribed developmental continuum. This lack of significant difference was obtained before and after a multimedia treatment in which both groups were trained to focus their attention on the different ways children moved. 78 Research Questions in the Development of Observation Systems A review of the literature supports the contention that the ability to observe and analyze movement is not a generic ability. With experience and training most individuals can improve their observational skills. Skill in observation and subsequent evaluation is enhanced if it is based on a behavioral framework around which a functional classification system can be established. Bayless (1981) has demonstrated an increased ability to rate motor performance when observers have available a predescribed checklist. Hoffman (1977b) suggested that systematic taxonomies which assist the teacher in clearly identifying correct and incorrect sport skill performance should be developed. Developmental progress in fundamental motor skills would probably be more easily categorized through the use of developmental sequences (Roberton, 1977a). Regardless of the behavioral framework chosen, Fishman and Anderson (1971) indicated that researchers must consider the complexity of the observational system they design: 1. Can it be used in real life situations or does it require a permanent record, such as an audiotape or videotape? 2. Can a single observer use the system and obtain 79 accurate and reliable records, or is a team of observers necessary? 3. Does the recording instrument provide for simple or complex coding schemes? These questions can be extended to include: 4. In what settings is a given observational system to be employed? 5. What are the necessary optimal and/or minimal conditions for using a particular categorical system in an observational setting? 6. Do these conditions change with different groups of observers? 7. What is the best method for exploring the answers to these questions? While Robinson (1974) began exploring some of these questions in relation to developing motor patterns, this review of literature indicates that most of the research conducted on developmental motor sequences has taken place since the Robinson study. Very little of this research has examined the reliability of developmental sequences as observational tools. Advances in generalizability theory (Shavelson & Webb, 1981) have provided a method by which to ’explore questions about the complexity, reliability, and optimal/minimal conditions of observational systems. The studies conducted by Mosher and Schutz (1983), and by Ulrich 80 et al. (1988) are the first in which generalizability theory was used to examine developmental sequences in relation to the conditions of observation. Generalizability Theory Historically, the measurement of sport skill performance has been associated with standardized tests (Godbout & Schutz, 1983). Product-moment correlations have traditionally been employed in determining the reliability of norm-referenced instruments and have been calculated using either test-retest scores or split-half scores (Baumgartner, 1969). These bivariate statistics examine the variance between pairs of scores rather than the variance between individual scores within the same class of measurement (Kroll, 1962). The introduction of intraclass correlation techniques extended the estimation of reliability beyond the limitations of two parallel sets of scores (Baumgartner, 1969: Ebel, 1951: Medley & Mitzel, 1958). The intraclass correlation coefficient is based upon an analysis of variance within univariate measures. It expresses measurement error as the relationship between true score variance and observed score variance. In its classical interpretation, the intraclass correlation coefficient 81 reflects the true variance for a population of scores in proportion to undifferentiated error variance. Determining test reliability using either bivariate statistics or the intraclass correlation technique has been limited by the assumption that the observed performance variability is attributable only to individual differences among subjects and to undifferentiated measurement error (Booth, Mitchell, & Solin, 1979). Practical application of classical measurement theory actually results in multiple sources of measurement error. These sources vary according to the data-collection design and thereby affect the estimation of the reliability coefficient. This would be particularly true with observational measurement systems that employ human raters in place of standardized, quantitative assessment instruments. Rarely are human raters equivalent in their observational skills, nor are the conditions of measurement (such as the number of items on a test or the number of performance trials) equivalent between observational sessions (Frick & Semmel, 1978). The reliability of an observational system, therefore, refers to the consistency with which a group of observers using the system can measure individual differences in performance under varying conditions of measurement. The classical interpretation of reliability limits the generalizability of the measurement instrument to the conditions in force at the 82 time of testing. In addition, it does not allow for differentiating between possible sources of measurement error. Cronbach and his colleagues (Cronbach, Gleser, Nanda, & Rajaratnam, 1972: Cronbach, Rajaratnam & Gleser, 1963: Gleser, Cronbach, & Rajaratnam, 1965) have reinterpreted reliability theory as generalizability theory. Generalizability theory is concerned with delineating the observed score variance into more than one source of measurement error. The theory asks to what extent an estimated true score, known as a universe score in generalizability theory, can be generalized to the variable conditions of measurement. Since the universe of measurement conditions is subject to change, a reliability correlation that is based solely on performance error is inadequate as an indicator of measurement precision. Generalizability coefficients, on the other hand, reflect the magnitude of each source of error defined in the measurement design. As the design changes, the magnitude of error changes, and so does the generalizability coefficient. In order to account for multiple sources of conditional error, Cronbach et al. (1972) expanded upon the techniques used in the computation of intraclass correlation coefficients. Multiple sources of error are defined by the context of the measurement conditions, such as the 83 measurement instrument used, the number of trials performed, or the observational ability of the tester. Through analysis of variance, one can calculate variance components to estimate the variability attributable to each error source. Whereas the conventional F-statistic obtained from an analysis of variance focuses on whether a variable has made a significant contribution to an obtained score, the variance components derived from the same analysis reflect the size of that contribution (Berk, 1979). Shavelson, Webb, and Rowley (1989) state: "GT is to measurement what the ANOVA is to substantive research. Just as the researcher attempts to identify and estimate the effects of potentially important independent variables, G theory attempts to identify and estimate the magnitude of the potentially important sources of error in a measurement." The analysis of variance procedure, therefore, is a tool for performing a generalizability analysis, but is not the significant feature of generalizability theory. Each variance component in the ANOVA design can be examined for its contribution to the observed score variance. Decisions can then be made regarding methods to reduce error variance, such as (a) reducing or increasing the number of observations under a particular measurement condition, (b) selecting and controlling some dimension of a measurement condition, or (c) ignoring a measurement condition. 84 Moreover, generalizability coefficients for varying measurement conditions can be computed from algebraic combinations of the variance components. The resulting coefficients can then be compared to determine whether certain conditions are more optimal than others for achieving a desired level of precision in the measurement procedure (Taylor, 1979). Cronbach et al. (1972) see the question of reliability as a question of the accuracy with which performance scores can be generalized to a universe of measurement conditions. Adjustments in the measurement conditions can be made more reliably when the sources of error are known. Shavelson et al. (1989) note that just as factorial ANOVA extends simple ANOVA procedures and allows the researcher to address more complex questions of multiple variable effects, generalizability theory extends classical reliability theory and allows the examiner to acknowledge "multiple influences on measurement variance". For example, in classical theory test-retest reliability limits measurement error to either day-to-day variation or test item sampling, while alternate-forms reliability confounds the two sources of error. Generalizability theory acknowledges each source of error separately and allows the magnitude of each source to be estimated. 85 Generalizability theory has been acknowledged as providing a more comprehensive, coherent, and flexible framework than classical reliability theory in describing the consistency with which a measurement instrument is employed (Berk, 1979: Cardinet, Tourneur, & Allal, 1976, 1981: Rentz, 1980: Shavelson & Webb, 1981). In comparing 16 indices of interobserver agreement, Berk (1979) cited eleven advantages for using a generalizability coefficient to report interobserver reliability. He noted that generalizability theory provides a comprehensive analysis of the factors related to the reliability of observers and can be applied to a variety of categorical and quantitative behavioral observational systems. Several theoretical contributions to generalizability theory have been presented since Cronbach and his associates formulated the structure of the theory in their 1972 monograph (Shavelson & Webb, 1981: Shavelson, Webb, & Rowley, 1989). Cardinet et al. (1976) suggested that generalizability theory is not limited to an asymmetric measurement design preoccupied only with the differentiation of the subjects measured. A symmetrical design would allow researchers to define any of the variable conditions as the object of measurement. In order to clarify their viewpoint. Cardinet et al. (1976, 1981) introduced several new concepts to generalizability theory. Among these concepts are a 86 four-phase design framework for conducting a generalizability analysis, the use of a Venn diagram for depicting the mean squares and the variance components in the factorial design, and a list of all possible differentiation problems for a three-dimensional design. Rentz (1980) has also added to the comprehensibility of generalizability theory by providing some rules of thumb for the computation of variance components and generalizability coefficients. Taylor (1979) and Godbout and Schutz (1983) have provided new interpretations for computing inter-rater generalizability, intra-trial generalizability, and an overall index of score generalizability. Despite these simplifications in several of the theoretical aspects, the terminology remains confusing and therefore merits further attention. Terminology Some of the terminology used in generalizability analyses are presented at this time. The definitions reflect contributions made to the theory by several authors (Brennan, 1983: Brennan & Kane, 1979: Cronbach et al., 1963: Cronbach et al., 1972: Cardinet et al., 1976, 1981: Godbout a Schutz, 1983: Gleser et al., 1965: Shavelson & Webb, 1981: Shavelson et al., 1989: Stamm & Moore, 1980: Taylor, 1979). 87 Generaliggbility (G) Studies. G studies estimate the magnitude of each source of measurement variance. Generally, G studies are associated with the design of a measurement procedure. Qecision (D) Studies. Cronbach et al. (1981) refer to decision studies as studies of optimization. The questions asked about the data collection procedures employed in a G study are addressed through the implementation of D studies. D studies allow the examiner to make decisions about what constitutes optimal measurement conditions or to draw conclusions about the reliability of data obtained under similar measurement conditions. Universe. Universe is distinguished from the term "population" in generalizability theory. Whereas population refers to the objects of measurement,,such as subjects or test objectives, universe refers to the conditions under which measurement occurs, such as occasions or observers of test items. Univerpe of Admissiple Conditions. The universe of admissible conditions includes all possible observational conditions encompassed in the generalizability study. gpivergp of Generalization. The universe of generalization refers to the set of conditions employed in a decision study, in other words the conditions of measurement to which.measurement decisions are generalized. This 88 universe consists of any subset of the universe of admissible conditions. qugp. The term "facet" replaces the term "factor" associated with analysis of variance. The facets are those aspects of the measurement procedure for which variations are to be studied. Any single facet is a set of similar conditions of measurement, such as subjects, test items, or performance trials. The number of conditions within the set constitutes the number of observations for the facet. Varignce Copponents. Each facet in the factorial design can be designated by a distribution of scores with a mean of zero. The variance of the distribution is called the variance component for that facet. Variance components are analogous to the group effects in a fixed effects analysis of variance. The sum of the variance components is analogous to the total sum of squares. PrincipIe of Symmetry. The principle of symmetry affirms that each facet of a factorial design can be selected as the object of study. The statistical operations defined for one facet may be transposed when a different facet becomes the focus of measurement. _gce of Differentiation. The face of differentiation is defined by the set of subjects, objects, or characteristics to be compared. Traditionally, the field of psychometrics has used subjects' scores as the object of 89 measurement, differentiating on the variability of subject performance. The principle of symmetry, however, allows the objects of measurement to vary with respect to the measurement problem. For example, a study of educational survey instruments may differentiate on educational objectives, whereas a study of student progress or development may differentiate on the occasions of measurement over time. Face of Generalization. The face of generalization is defined by the conditions of the study that affect the measures taken. These facets of generalization contribute to measurement error both systematically and randomly. Cardinet et al. (1981) suggested that measurement conditions are actually the instrumentation of the data collecting process and preferred to designate these facets as the fggp of instrumentation. The terms are interchangeable. Universe Score. The universe score is equivalent to the true score in classical reliability theory. It represents the expected mean score from a sample of obtained scores. Because the universe of measurement conditions to which scores can be generalized is subject to change, the term "true score" is not an appropriate representation of the sample mean. Universe score more suitably emphasizes the diversity of conditions to which generalization may 9O occur. The face of differentiation is represented by the universe score. sterved Score. The observed score is known traditionally as the score obtained during the observational procedure. The variability found in a sample of observed scores includes both random and systematic measurement error. Since it is subject to change from one application of the design to another, the observed score is an estimated parameter in generalizability theory. It consists of both the face ofgeneralization and the face of differentiation. Generalizability (G) Coefficient. The G coefficient is analogous to the reliability coefficient in classical theory. It is defined by the ratio of the universe-score variance to the expected observed-score variance, therefore it is determined by the facets of differentiation and the facets of generalization employed in the measurement design. Unlike the classical reliability coefficient, the coefficient of generalizability varies as the universe of generalizability varies. Framework for Conducting a Generalizability Analysis Cardinet et al. (1981) have clarified the differences between the roles of the analysis of variance and the generalizability computations by presenting a four phase 91 framework for generalizability analyses. The first two phases are concerned with the analysis of variance and the last two phases with generalizability concepts. A brief description of each phase follows. Eggpep; -- opservation. The observational design establishes the conditions of measurement and the relationships among those conditions. A fully crossed design that incorporates a number of facets is desirable as it allows for a number of different measurement designs to be employed in the generalizability analyses (Kane and Brennan, 1979: Shavelson et al., 1989). Under some measurement circumstances, however, nested and confounded variables are determined in this early phase of a study. An analysis of variance procedure contributes to the calculation of mean squares for each source of measurement variation. gMgpe 27--p§§timation. The estimation design specifies the appropriate analysis of variance model to be used in estimating the variance components for each facet in the observational design. The facets are designated as either fixed or random based on the sampling procedures employed. This designation determines whether the design is a mixed model or a random model, which in turn determines the calculations used for obtaining variance components. 92 Phage 3 -- Measurement. Based on the universe of admissible objects and conditions, the measurement design specifies the facets of differentiation (the objects of measurement) and the facets of instrumentation (the conditions to which measurement is generalized). The variance components for these facets, which are estimated during phase 2 of a study, are combined appropriately to obtain universe score variance and error variances. From these variances the generalizability coefficient can be calculated. Eggse 4 -- Optimigation. The fourth phase entails decision studies in which modifications are made in the facets of differentiation and the facets of instrumentation. These modifications reflect alterations in the universes of generalization, such as changing the number of items tested or increasing the number of observers employed in the study. They are aimed at optimizing measurement precision. This phase of the analysis results in different universe score variances, error variances, and generalizability coefficients for each modification undertaken. 93 Application of Generalizability Analysis to Motor Performance Assessment This review has emphasized that observing and interpreting motor behavior is critical to developing teaching strategies aimed at improving children's motor skills. The observation and assessment of motor performance occurs through both quantitative and qualitative measurement. Generalizability analysis offers a flexible method by which to estimate the reliability of measurement systems for assessing motor performance. Berk (1979) has noted that it can be used to analyze both categorical and quantitative observational systems. The ease with which a generalizability study can be modified to account for the reliability of observations under various conditions of measurement reinforces the advantages of the methodology in determining the minimum conditions of observation needed to obtain reliable motor performance scores. The appropriateness of using generalizability analyses in physical education research has been acknowledged (Godbout & Schutz, 1983: Safrit, Atwater, Baumgartner, & West, 1976: Stamm & Moore, 1980). Physical education researchers, however, are only just beginning to employ a generalizability framework in reliability studies. One of the first studies reported in the physical education 94 literature to use a generalizability analysis did so to establish the validity of a systematic observational instrument (Physical Education Observation Instrument) for coding the interactive behaviors of teachers and students in an instructional setting (Taylor, 1979). Clark, Stamm, and Urquia (1979) employed a generalizability framework to examine the reliability of children's performance on a balancing task. They determined that six trials on a single day would provide reliable performance measures of the task, despite the "inherent variability" of children's developing motor performance. Ulrich, Riggen, Ozmun, Screws, & Cleland (in press) reported the results of novice and competency-based trained observers in assessing three age-appropriate sport skills performed by children with mild mental retardation. Three important implications can be derived from their study: (a) different conditions of measurement are necessary for different skills: (b) competency—based training in visual observation and analysis contributes to the economical use of available assessment time in a clinical setting: and (c) the standardization of assessment conditions is facilitated through a generalizability analysis. Only two studies have applied generalizability theory to rating the qualitative development of gross motor skills. Mosher and Schutz (1983) examined the reliability and 95 practicality of a test of overarm throwing. Their measurement instrument was based upon a modification of the foot-placement, body-rotation, and arm-action developmental sequences for overarm throwing proposed by Roberton (1977b, 1978a). They found the foot-placement and the body-rotation sequences to be reliably scored by a single observer with minimal training. The arm-action sequence, however, was more difficult to score. Ulrich et al. (1988) determined the minimal conditions of observations needed to obtain generalizable results for the hop, jump, and run. The developmental sequences used for rating the subjects were the composite configurations proposed at Michigan State University (Haubenstricker et al., 1984: Haubenstricker et al., 1975: Seefeldt et al., 1972). Their results indicated that the run and the jump were not reliably rated by one observer in five trials or less. In contrast, the hop was reliably rated by one observer viewing three performance trials. The researchers note that minimal training was given to the observers in order to familiarize them with the sequence descriptions. They suggest that the reliability of rating motor skills could be enhanced through competency-based training in movement skill analysis. The work of Mosher and Schutz (1983) and of Ulrich, Ulrich, and Branta (1988) demonstrate how generalizability 96 analysis can be used to determine the usefulness of developmental sequences. Ulrich, Riggen, Ozmun, Screws, and Cleland (in press) have demonstrated how generalizability analysis can be used to determine differences in the observational abilities of two groups of observers. Together these studies offer a research paradigm for comparing the reliability and practicality of discrete developmental sequences proposed for the same skill as well as the reliability and practicality of the sequences when used by different categories of observers. These were the issues of concern in the present study. Ulrich et al. (1988) have demonstrated that graduate students in motor development and adapted physical education classes can reliably rate the hopping performance of children when using a composite developmental sequence. The purpose of this study was to expand upon their work in several directions. The reliability of the composite developmental sequence for hopping (Haubenstricker et al., 1975) was again examined, as well as the reliability of the arm sequence and the leg sequence proposed by Halverson and Williams (1985). The minimum conditions of observation suggested in the application of these sequences were compared. The ability of undergraduate physical education students and elementary education students to apply these sequences in rating the hopping performance of children was 97 also differentiated. Prior to describing the methodology and results of this study, however, a comprehensive review of the development of hopping is appropriate. The Development of Hopping The mastery of hopping during the childhood years is an important event. Although Wickstrom (1983) noted that hopping has limited use as a form of locomotion, he also identified the more readily apparent incorporation of this fundamental motor skill into a variety of movements and activities. Once hopping fundamentals are mastered, children will challenge their own ability by moving sidewards, backwards, or turning and by incorporating the newly developed skill into common childhood games such as hopscotch and jump rope (Espenschade & Eckert, 1980: Eckert, 1987). Hopping skill can be studied from a number of perspectives. Hopping can be viewed as an independent fundamental skill or in combination with other movement patterns. An individual may perform a single hop or several continuous hops. When performed continuously, hopping may or may not have a rhythmic quality. An individual can hop in one place or by moving to a new location (Williams, 1983). 98 Hopping is an important antecedent skill in the attainment of many locomotor and manipulative sport and dance skills. Locomotor patterns involving two or more movements on the same foot before weight is transferred to the other foot require hopping. The fundamental motor skill of skipping is the first of such "double-task" patterns to develop, followed by an even—rhythm step-hop—step-hop pattern (Roberton & Halverson, 1984). Used in both even and uneven rhythmic patterns, the hop is an important component of many common dance steps such as the schottische, the polka, and the mazurka. Sport skills occasionally contain a hop in their movements patterns, often to dissipate force or to control movement in sudden stops or changes of direction. In their mature forms, the fundamental manipulative skills of kicking and punting use a hop to dissipate the force imparted during the propulsive phase of the kick or the punt. The triple- jump in track and field and the lay-up shot in basketball incorporate the hop for force production. Finally, gymnastics stunts may use hopping for force production, force dissipation, or locomotion. Juxtaposed with the renewed interest in describing developmental motor sequences, systematic research in hopping has recently taken place. The description of hopping in both qualitative and quantitative terms is now 99 found regularly in motor development texts (Cratty, 1975, 1979: DeOreo & Keogh, 1980: Espenschade & Eckert, 1980: Eckert, 1987: Godfrey & Kephart, 1969: Haywood, 1986: Keogh & Sugden, 1985: Payne & Isaacs, 1987: Williams, 1983). Both a discussion on the definition of hopping and a review of the hopping research literature are in order. Defining a Hop Motor development specialists agree that hopping involves taking off and landing on the same foot. Rather than defining hopping as an independent fundamental skill, however, many current specialists refer to hopping as a version of, type of, or specialized form of jumping (Espenschade & Ekert, 1980: Godfrey & Kephart, 1969: Haywood, 1986: Payne & Isaacs, 1987: Wickstrom, 1983). While it is true that hopping and jumping are concerned with both vertical and horizontal body projection into space, the physical description of the spatio-temporal relationships between body segments is not the same (Williams, 1983). The confounding of hopping and jumping could possibly have its origins in the work of McCaskill and Wellman (1938) and Gutteridge (1939). In describing the motor achievements of young children these researchers consistently referred to hopping on "both feet" as part of the order of tasks leading 100 to a single-foot hop. This series of jumps that precedes the ability to hop might be better described as "bounce jumps" (Roberton & Halverson, 1984). Considering the distinction between inter- and intra- developmental sequences (Roberton, 1978c), and considering that hopping is not a combination task such as skipping, it would seem more appropriate to define hopping without using jump in the definition. Roberton and Halverson (1974) identified the following five ways in which an individual can travel from one place to another by transferring weight between the feet: 1) from one foot to the other: 2) from one foot to the same foot; 3) from one foot to two feet: 4) from two feet to two feet: 5) from two feet to one foot. Whereas jumping involves a propulsive force from either one foot or both feet and landing on two feet, hopping is identified by taking off and landing on the same foot. The following definition of hopping by Broer and Zernicke (1974) was employed in this study: "When the body is projected into the air by the propulsive force of one foot and the subsequent landing is on the same foot, the action performed is a hop." 101 As hopping does require the strength and balance to project the body into the air and to land without falling, jumping is an obvious antecedent skill to hopping. McCaskill and Wellman (1938) noted that the median age for continuous jumping of one to three jumps occurs at 38 months, while the ability to hop one to three hops does not occur until 43 months. escriptive Research on Hopping Most of the historic research on hopping has resulted in quantitative measures of distance, time, and accuracy. The age at which children can first perform one or more hops was noted by researchers interested in the early developmental years (Bayley, 1935: Frankenburg & Dodds, 1967: Gesell, 1940: McCaskill & Wellman, 1938). Researchers examining hopping ability at the preschool and primary school years have recorded the time it took children at different ages to perform a set number of hops (Denckla, 1974: Touwen, 1979) or the time it took to hop a given distance (Jenkins, 1930: Keogh, 1965: Seils, 1951). Several physical educators included hopping items in skill test batteries devised to test for neuro-muscular skill capacity (Johnson, 1932) or general motor ability (Carpenter, 1942). 102 Several studies describing the movement patterns of young children have proposed qualitative criteria for measuring success in hopping (Gutteridge 1939: Gesell, 1940: Godfrey & Kephart, 1969: Sinclair, 1974). Gutteridge (1939), for example, noticed performance differences between children with regard to balance, unnecessary movements, posture, ability to change directions, use of the arms, and ability to change feet. She defined hopping proficiency as achievement of the basic skill and she characterized proficient behavior by coordinated movements, easy performance with a display of satisfaction, and evidence of accuracy, poise and grace in performance. Sinclair (1974) was also interested in establishing identifiable characteristics of movement. She contended that dynamic balance, opposition and symmetry, total body assembly, rhythmic locomotion, agility, and postural adjustment were important elements in the development of mature hopping skill. While both Gutteridge and Sinclair used qualitative terminology to discuss developmental traits of hopping, neither individual attempted to isolate characteristic movements along a developmental continuum. More recently researchers have proposed developmental sequences for hopping (Haubenstricker et al., 1975: Roberton & Halverson, 1984: Gallahue, 1982: Halverson & Williams, 1985). These investigators have hypothesized 103 sequences based on age-related changes in the positioning of the body and the use of the limbs during continuous hopping over a distance. Haubenstricker et al. (1975) used high- speed film in a mixed longitudinal study to identify characteristic changes in hopping patterns across age. Their developmental sequence in hopping is a four stage composite approach. In order to describe hopping within his three phase developmental framework for fundamental motor skills, Gallahue (1982) condensed these four stages to three. Roberton and Halverson (1977) also used high-speed film to identify stages of hopping, but their descriptions of arm-and leg-movement patterns are framed within the component model for developmental sequencing. Their initial work was based on seven children filmed longitudinally over time. Using a prelongitudinal screening paradigm (Roberton, Williams, & Langendorfer, 1980), these sequences were recently tested on seventy-two children by Halverson and Williams (1985). Their data resulted in a revision of the original sequences. Halverson and Williams (1985) proposed four steps in the development of hopping leg action and five steps in the development of hopping arm action. Both product and process research have revealed gender differences in children's development of hopping. Keogh (1965) noted that girls hop sooner, faster, with better 104 quality, and in better control. Girls have been shown to be more advanced than boys in the age at which hopping begins (Frankenburg & Dodds, 1967), in the number of hops that can be performed without losing balance (McCaskill & Wellman, 1938), in the time it takes to cover a distance of 50 feet (Jenkins, 1930: Keogh, 1965: Seils, 1951), in the degree of proficiency obtained (Gutteridge, 1939), in the accuracy and control of movements (Keogh, 1965, 1970), and in the progress through developmental stages (Branta, Kiger, & Yager, 1985: Halverson & Williams, 1985: Haubenstricker et al., 1989). Williams (1983) stated that while both biological and sociological reasons have been proposed for these gender differences, the reasons given for differences in performance by girls and boys at any age are not clear at this time. Although biological maturity and more frequent practice appear to be appropriate explanations for girls' better performance in hopping, these explanations are only inferences gleaned from developmental research. A Compositeypsvelopmental Sequence for Hopping A developmental sequence for hopping was first proposed by Haubenstricker, Henn and Seefeldt (1975). Their work described four progressive stages in the total-body (composite) configuration displayed by children when 105 hopping. Tables 1 and 2 (pages 108 and 109) report the standards of performance in these stages obtained during the analysis of data from the Early Childhood Motor Skills Development Study at Michigan State University (Haubenstricker, et al., 1989). These tables list performance standards by age and gender for both the preferred and non-preferred foot. Validation of this 4-stage sequence has occurred through cross-sectional and mixed-longitudinal studies (Branta et al., 1984: Branta et al., 1985: Early Childhood. 1985: Haubenstricker et al., 1989: Way, Haubenstricker, & Seefeldt, 1979). Figures 1 and 2 (pages 110 and 111) illustrate the observed frequencies of performance at these four stages as analyzed from the Early Childhood data. According to the criteria for accuracy in developmental sequencing (Langendorfer, 1987: Halverson & Williams, 1985), the functional relationship of these frequencies validates the accuracy of the proposed stages. Figure 3 (page 112) displays film tracings of the developmental stages of hopping as filmed by Seefeldt and Haubenstricker and sketched by Kiger (1985). The developmental sequence for hopping as proposed by Haubenstricker, Henn, and Seefeldt (1975) is described as follows: 106 Stage 1: The non-support knee is flexed at 90 degrees or less with the non-support thigh parallel to the surface. This position places the non-support foot in front of the body so that it may be used for support in the event that balance is lost. The body is held in an upright position with the arms flexed at the elbows. The hands are held near shoulder height and slightly to the side in a stabilizing position. Force production is generally limited so that little height or distance is achieved in a single hop. Stage 2: The non-support knee is fully flexed so that the foot is near the buttocks. The thigh of the non-support leg is nearly parallel to the surface. The trunk is flexed at the hip resulting in a slight forward lean. The performer gains considerable height by flexing the hip joint. In addition, the thigh of the non-support leg aids in force production by flexing at the hip joint. Upon landing, the force is absorbed by flexion at the hips and the supporting knee. The arms participate vigorously in force production as they move up and down in a bilateral manner. Due to the vigorous action and precarious balance of 107 performers at this stage, the number of hops generally ranges between two and four. Stage 3: The thigh of the non-support leg is in a vertical position with the knee flexed at 90 degrees or less. Performers exhibit greater body lean forward than in stages one or two, with the result that the hips are farther in front of the support leg upon take-off. This forward lean of the trunk results in greater distance in relation to the height of the hop. The thigh of the non- support leg remains near the vertical (frontal) plane, but knee flexion may vary as the body is projected and received by the supporting leg. The arms are used in force production, moving bilaterally upwards during the force production phase. Stage 4: The knee of the non-support leg is flexed at 90 degrees or less, but the entire leg swings back and forth like a pendulum as it aids in force production. The arms are carried close to the sides of the body, with elbow flexion at 90 degrees. As the non-support leg increases its force production, that of the arms seems to diminish. 108 9.9 ..9 ..9 ..9 9.9 9.9 9.9 9.9 9.9 9.9 9.9 9.9 .9 .9 99. 9.99 999 9.99 9.9 9.. 9.. 9.9 9.9 9.9. 9.9. 9.99 9.99 ..99 9.99 9.99 8991!! 999 9.99 .99 9.99 9.9 9.9. 9.99 9.2 9.99 9.: ..99 9.99 ...9 9.99 9.99 9.9 . U999 99. 9.99 9.9 9.99 ..99 9.9 9.99 9.99 9.99 9.9 9.3 ..9. 9.9. 9.. 9.9 9.9 9 3999 9 99 9.9 99 9.. 9.99 9.99 9... 9... ... 9.9. ..9 9.. 9.9 9.. 9.9 9.9 . u9=e v 9 9.9 9 9.9 9.9 9.9 9.. 9.9 9.9 9.9 9.9 9.. 9.9 9.9 9.9 9.9 ugko .999. .99.. .999. .99.. .99. .99. .99. .99.. c... .99.. .99.. .99.. .99.. .99.. :2. .99.. .2. a a a a a a a z a a a a g .2. .s. .... :9. 99.99 99.99 99.99 .999 .999 99.99 2 95.3 9.9825999 .999. .999.& 9 .9909999 .99999999 .mucmum .uoxoauumcmnsem. mooeum nsoauo> be uoom ommuououm on» so Deanna: coupawnu no unmouom 9H OHQMB 109 ..9 ..9 9.9 ..9 ..9 9.9 9.9 ..9 9.9 9.9 9.9 9.9 ... .9 999 ..99 .99 ..99 9.9 9.9 9.9 9... 9.9. 9.99 9.99 9.99 9.99 9.9. ..99 9.99 99.99.! 9.9 9.99 999 9.99 9.99 9.99 9.99 9.3 9.99 ..99 9.99 9.99 9.99 9.99 9.9. 9.9 . w9=u 99. 9.99 99. ..9. ..99 9.99 9.99 9.99 9.99 9.99 9... 9.9 9.9 9.9 9.9 9.9 9 mgkn 9. 9.9 99 9.9 9... 9.9 9.9. 9.9 ... ..9 9.9 9.9 9.9 9.9 9.9 9.9. 9 «9:9 9 9.9 . ..9 9.9 9.. 9.. 9.9 9.9 9.9 9.9 9.9 9.9 9.9 9.9 9.9 9 p 89.9 .999. .99.. .999. .99.. .99. .99. .99. .99.. .9... .99.. .99.. .99.. .99.. .99.. .99.. .99.. .2. an» a a a a a a a a a a ..a ‘Bio .... .... .... ..9. 99.99 99.99 99.99 .999 .999 99.99 9912!. mil! 9.98.9393 .mmmH .uoowu w .muoxmum .uoaeuoem .mucmum .uexownumconscm. moomum naowuo> as boom peuuouenmlcoz on» so unfinaoz ceupawnu uo unmouem "N OHQMB 110 .mwma .mmowfl w "H musofim .muoxmum .upamumom .mucmum .uexowuumconsmm. mod mnouoc neomum Dawnnom no mean: How coconusouo no mmfiucusomuh pe>umnno 99 999 999 999 999 999 es: 99.99 99.99 09.99 99.99 9799 99.99 i 9.9 9.9 9.9 9.9 9.9 9.9 g P I. ll 9. a: 9 '9. .9. 00.9 ‘eJOee-De ~a 9999 9999 .9. 9.9 .9 . \ ...: . 0 \ 0‘ “‘0 . ... al.! 9 9| 9 II 9‘~9h 9\‘eee b \‘he. I .0 . eAe-‘OeEOEOUOd. 0“. \ 9&u .\ .\ [‘1 P‘ ‘\\\\ \\ l O. ‘9 I 9‘09 he“. \ 09' ” $9... \ . .. .. \ guy] 39. o. \ .l on I .39. \\ I ... I m. .\\¢£ / ... \ .l 99 .9597 I \ \ \ . I tin.‘ 99 D D ‘ E ‘I a: 95% “d. ' .O 955 99. won 9 t 0 '- 9eeeeefi‘r ".3 .... v:...\.:.: 999‘.‘ 1.9 9 a: e .9- .llisor. 19. ”>8 I 99 ozxcox .9. 99 .mwma .uwoflu w .muoxmwm .upauueem .mucoum .moxofiwumcensmz. 90¢ mmouoc meomum ocwndom uo mmamfiom you mocmuusooo no newucosumuh pe>uemno "a madman 99 99 99 9 =9 9.9 999 99“ 99.99 99.99 99.99 99.99 99.99 99.99 i 3 O .l 99 1 99 \?:.: 1. 99 1 1 1 .I 99 9 .98 . t. 9 999.9 . E 9 a! e U I 99 9 1 e .9. 8119.99 .... 99 a: r! .5 T 99 Figure 3: Film Tracings of the Developmental Sequence of Hopping as Filmed by Seefeldt and Haubenstricker (Kiger, 1985) 113 Component Developmental Squsnces for Hopping Roberton and Halverson (1977) used a component model of motor task development (Roberton, 1977a) in describing developmental sequences for hopping. They identified sequential development for arm action and for leg action. These sequences were revised by Halverson and Williams (1985) during a prelongitudinal screening for sequence comprehensiveness and developmental accuracy. Evidence of developmental accuracy in both the proposed arm sequence and the leg sequence were displayed in the sequential ordering of the frequency distributions obtained for the developmental steps and in the appropriately increasing and decreasing signs of the functions. The description of these sequences follows: Leg Action. Step 1: Mgmgntary fligMp. The support knee and hip quickly flex, pulling (instead of projecting) the foot from the floor. The flight is momentary. Only one or two hops can be achieved. The swing leg is lifted high and held in an inactive position to the side or in front of the body. 114 Step 2: Fall and catch: Swing leg inactive. Forward lean allows minimal knee and ankle extension to help the body "fall" forward of the support foot and, then, quickly catch itself again. The swing leg is inactive. Repeated hops are achieved. Step 3: Projected takeoff: Swingslsg assists. Perceptible pre-takeoff extension occurs in the support leg, hip, knee, and ankle. There is little delay in changing from knee and ankle flexion on landing to takeoff extension. The swing leg now pumps up and down to assist in projection, but range is insufficient to carry it behind the support leg. Step 4: Projection delay: Swing leg leads. The weight of the child on landing is smoothly transferred along the foot to the ball before the knee and ankle extend to takeoff. The range of the pumping action in the swing leg increases so that it passes behind the support leg when viewed from the side. Arm Action. Step 1: silsteral inactive. The arms are held bilaterally, usually high and out to the side, although other positions behind 115 or in front of the body may occur. Any arm action is usually slight and not consistent. Step 2: Bilsteral reactive. Arms swing upward briefly, then are medially rotated at the shoulder in a winging movement prior to takeoff. It appears that this movement is in reaction to loss of balance. Step 3: Bilateral assist. The arms pump up and down together, usually in front of the line of the trunk. Any downward and backward motion of the arms occurs after takeoff. The arms may move parallel to each other or be held at different levels as they move up and down. Step 4: Semi-opposition. The arm on the side opposite the swing leg swings forward with that leg and back as the leg moves down. The position of the other arm is variable, often staying in front of the body or to the side. Step 5: Opposingiassist. The arm opposite the swing leg moves forward and upward in synchrony with the forward and upward movement of that leg. The other arm moves in the direction opposite to the action of the swing leg. The range of movements in the arm action may be 116 minimal unless the task requires speed or distance. Roberton and Halverson (1984) have examined the relationship of the developing body components in hopping through profiles of the component sequences. They stated that certain combinations of steps in the arm and leg sequences for hopping are more likely to be seen together: ”We have noted that Step 1 (momentary flight) in the legs and Step 1 in the arms (bilateral inactive) always occur together. Also, the most advanced hopping combines the most advanced steps in both components." (pp. 64-65). Most recently, Roberton and Halverson (1988) have employed a dynamical systems approach to examine the developmental sequence, relative timing, center of gravity, and phase- plane analyses of longitudinal data of the development of hopping. Their data support evolving relationships between the separate components of the body that ultimately are evidence in developmental profiles for arm and leg action. As Flavell (1972, 1985) and Brainerd (1978) have emphasized, the specification of steps along a developmental continuum is an arbitrary process defined by the perspective of the sequence creator. Several motor developmentalists (Gallahue, 1982: Wickstrom, 1983: Williams, 1983) have defined their own hopping sequences, all of which bear face validity to either the Haubenstricker et al. (1975) 117 composite sequence or the Roberton and Halverson (1977, 1984) component sequences. Although each sequence provides keywords that are aimed at assisting an observer in classifying children's hopping performance according to developmental steps or stages, surprisingly little information is available on the reliability with which individuals are able to do so. Halverson and Williams (1985) reported 90% interobserver and 90% intraobserver agreement in categorizing the arm action component for hopping, as well as 95% interobserver and 100% intraobserver agreement in categorizing the leg component. Ulrich et al. (1988) have recently employed generalizability analysis in examining the reliability with which students in graduate physical education classes could rate hopping performance according to the Haubenstricker et al. (1975) composite sequence. The reliability with which different groups of observers are able to classify hopping performance using different models of developmental sequences, however, has not been studied. The implications of such information in determining minimal and/or optimal conditions for analyzing children's hopping behavior seems readily apparent, particularly for training individuals in the assessment of children's motor skills. Before such a study can be undertaken, however, the ages across which hopping develops must be clearly identified. 118 The Developmental Ages of Hopping The first attempts to hop take place between 3 and 3 1/2 years of age (Frankenburg & Dodds, 1967). McCaskill and Wellman (1938) cited 43 months as the average age at which children are able to perform one to three hops, while Bayley (1935) cited 49.3 months for the same performance. Defining success in hopping as four hops on one foot, Sinclair (1974) noted that most 3-year-old children are "inept" at hopping, but by 4 years of age 67% of children are able to hop. Clearly most children are still beginning hoppers at age 4. Touwen (1979) noted that children between 4 and 5 years of age are capable of performing an average of five to eight hops. According to data obtained by both Touwen (1979) and Denckla (1974), children between 5 and 6 years of age can hop from nine to twelve times on one foot, whereas 6 year old children average between 13 to 25 hops. The Touwen data indicated that at 7 years of age only 25% of children are able to perform 20 consecutive hops. Denckla, however, discovered that by 8 years of age 80% of her sample could successfully perform 50 hops in place on either foot. The age at which children can hop continuously on one foot across a specified distance has also been observed (Jenkins, 1930: Keogh, 1965: Seils, 1951). Both Jenkins (1930) and Keogh (1965) found that a large number of 5-year- 119 old children were unable to complete a 50-foot hopping task without losing their balance. For 5-year-old girls the percentage unable to hop 50 feet was between 18 and 19 percent. The percentage of boys was even higher--31% in the: Jenkins sample and 38% in the Keogh sample. Even at 6 years of age, 34% of the boys in the Keogh sample were unable to complete the distance. Jenkins suggested that the a 50-foot hopping task would be too difficult for children younger than 5 years of age and that reducing the distance may augment children's ability to perform more trials. Eighty percent of the failures in her sample were from the age 5 group. She noted that at 5 years of age children are just beginning to maintain their balance while in motion. Children's proficiency in hopping was described rather globally by Gutteridge (1939). She discovered that only 33% of 4- to 5-year-old children were proficient at hopping. She described the average 4-year-old as awkward. Between 5 and 6 years of age children were capable of an "easily coordinated performance", and by 6 1/2 years of age 90% of children were proficient "hoppers". Improvement in the time it takes to hOp 50 feet appears not to occur in a linear fashion, but rather in performance spurts at different ages. Keogh (1965) found the greatest improvement in time between ages 6 and 7 for girls, while for boys the greatest improvement was from ages 7 to 8 and 9 120 to 10. Keogh's results for 5- and 6-year-old children were similar to those obtained by Jenkins (1930). Both Keogh (1965) and Jenkins (1930) noted that the variability in hopping performance is large between 5 and 7 years of age, especially for boys. Although several researchers have suggested that most children have achieved basic, proficient, or mature patterns of hopping by 6 years of age (Gutteridge, 1939: Sinclair, 1973: Williams, 1983), more recent work on the developmental patterns of hopping provide evidence that children may continue to improve in their ability to hop well beyond the preschool years (Branta et al, 1984: Branta et al, 1985: Early Childhood, 1985: Halverson and Williams, 1985: Haubenstricker et a1, 1989: Seefeldt & Haubenstricker, 1982: Way et al, 1979). The 5-year-old children in the Halverson and Williams study (1985) were rated at early or intermediate developmental steps in both arm action and leg action. Data from the Early Childhood Motor Skills Development Study at Michigan State University (1985) revealed that 13.2% of 5-year-old boys and 5.9% of 5-year- old girls were still hopping in a stage one pattern on their preferred foot. In this same age group, only 2.9% of the boys and 5.9% of the girls were capable of hopping in a mature stage on their preferred foot. Early reports of the ongoing research by Seefeldt and Haubenstricker (1982) 121 indicate that only 60% of the girls in their study were hopping in a mature stage by 7 years of age. Sixty percent of the boys did not achieve a mature hopping pattern until 7.5 years of age. The performance standards for hopping on the preferred foot reported by Way et a1. (1979) included data on children ages 72 to 107 months. At 8 1/2 years 61% of the boys and 55% of the girls were still not hopping in a mature pattern. In fact, approximately 4% of the boys and 4% of the girls were still performing a stage one hop at this age. Generally, increasingly mature patterns of hopping become more pronounced in 8-year-old children. It would seem that endurance, accuracy and control would improve as well. Testing children from kindergarten to sixth grade on both a 50-foot distance hop for time and a mat hop for accuracy, Keogh (1965, 1970: Cratty, 1975) indicated that 94% of the 8-year-old boys and 95% of the 8-year-old girls had the ability and the endurance to complete the 50-foot hopping task. The mat hop was fashioned after a similar test by Carpenter (1942). It required accuracy and control in hopping forward, diagonally, and sideward into specifically defined squares on a mat. Keogh found that hopping sideward was the most difficult of the tasks and that it did not discriminate between children until age 8 or 9. Additional work by Keogh (1970) on movement control 122 revealed that alternating from foot to foot while hopping in a 2-2, 3-3 or 2-3 rhythm did not occur with precision until approximately age 8 (Cratty, 1975). The data indicate that the development of hopping spans a much broader age range than was previously believed. To ascertain the reliability of observers in classifying immature through mature performance of children hopping, it is necessary to choose subjects from an age range that encompasses all of the developmental steps or stages. Additionally, to obtain several trials from children performing in immature stages it is necessary to specify a distance that provides for reasonable success. Halverson and Williams (1985) filmed children ages 3 to 5 performing five trials on each foot for a distance of 12 feet. Based upon this study and the preceding review of literature on hopping, a reliability study of the nature suggested should include children at least 3.5 years of age up to at least 8 years of age. Successful performance by this age group of children would probably be displayed over a distance of no less than 12 and no more than 30 feet. 123 Summary This review of literature has presented the research in and the issues surrounding the description of qualitative changes in children's developing motor skills. The theoretical differences between developmental sequences and developmental stages were discussed from two perspectives: _(a) a Piagetian view in developmental psychology, and (b) a classical, maturational-based view in motor development. The current conceptualizations of sequencing techniques in the description of fundamental motor skill development were examined and the research methodology for validating hypothesized motor sequences presented. Although researchers have advocated the application of developmental motor sequences in the form of observational assessment systems that can be used in the instructional process, the literature reveals that very little research has been conducted in this area. Not only do developmental motor sequences need to be validated longitudinally, but the ability of observers to employ these sequences reliably and accurately in rating the developing movement patterns of children needs to be established. Additionally, the minimal and optimal conditions of measurement in which reliable assessment can occur should be identified. This information 124 will contribute to the standardization of both research methodology and classroom assessment procedures. Generalizability theory was introduced as a research design and a statistical framework by which the reliability of observational systems can be examined. Physical education researchers have just begun to use generalizability analyses in reliability studies. It appears that this type of analysis serves as an excellent means of establishing the usefulness of developmental motor sequences to the practitioner. The current study investigated the usefulness of two different developmental sequencing techniques for rating hopping skill. As background for the data collection procedures used in this study, the descriptive literature on the development of hopping was presented. Conclusions were drawn about the ages of the children considered for the study, as well as the distance the children were asked to hopped. CHAPTER III METHODOLOGY The purpose of this study was to examine the generalizability of undergraduate kinesiology students and undergraduate elementary education students in assessing the hopping performance of children. Hopping performance was evaluated using developmental sequences that describe changes in the configuration of the total body and changes in the configuration of body parts. The observers rated the hopping performance of children both before and after receiving training in identifying the developmental level of a child's performance according to specific changes in body configurations. Measurement error was examined and compared for these pre-training and post-training assessments. Three developmental sequences that describe the changing body configurations during the development of hopping skill served as the measurement instruments in this study. The developmental sequence describing changes in the configuration of the total body was that hypothesized by Haubenstricker, Henn, and Seefeldt (1975) and prelongitudinally validated by Haubenstricker et al. (1989). The two component developmental sequences, one describing 125 126 changes in the configuration of the arms and one describing changes in the configuration of the legs, were those hypothesized and prelongitudinally validated by Halverson and Williams (1985). The study also investigated the minimal conditions required to achieve reliable results for each of the sequences when they are employed in the assessment of hopping by the two different categories of observers. All analyses were conducted under the framework of generalizability theory as conceived by Cronbach and his associates (Cronbach et al, 1963, 1972) and interpreted by Brennan (1983). Subjects The subjects for this study consisted of twenty boys and girls between 3.5 and 8.5 years of age. The nature of this study suggests that a continuum of hopping development from immature to mature patterns of movement should be assessed. The preceding review of literature has established that ages 3.5 to 8.5 encompass the years in which a wide range of hopping ability is displayed. The subjects were drawn from a larger sample of children enrolled in summer activity programs. Part of this larger sample was gathered from children enrolled in summer 127 motor skill programs and the summer laboratory school program at a major midwestern university. The remainder of the sample consisted of children enrolled in a summer Y.M.C.A. program in a neighboring community. The procedures for obtaining an expedited review from the human subjects committee were followed. A letter explaining the purpose of the study was mailed to the parents of children enrolled in the summer motor programs. Informed written consent was obtained from the parent(s) or guardian(s) of the children participating in the study (see Appendix A). The hopping performance of 75 children for whom written consent had been received was videotaped. Because the summer programs were organized according to age-level groupings, the original videotaped sample of children ranged in age from 3.5 years to‘9 years. The videotaping procedures are described later in this chapter. Sixteen subjects were eliminated from the pool of videotaped children for the following reasons: (a) inability to hop (n=8), (b) poor performance effort (n=2). (c) clothing (n=2) or a casted limb (n=1) that restricted full view of limb movements, and (d) errors in videotaping that erased parts of trials (n=2). Subsequently, the number of subjects in the videotape pool was reduced to 59. Twenty children were randomly selected from the videotape pool as subjects for a data-collection videotape. 128 This selection was based on a stratified sample according to age of subjects. Performances on the data-collection videotape were observed for the generalizability analyses conducted in this study. From the 39 individuals remaining in the videotape pool, selected performances were chosen for the six segments of a training videotape that was used to instruct observers on the developmental sequences for hopping. Observers Ten upper-division, undergraduate kinesiology students and ten upper-division, undergraduate elementary education students served as the observers in this study. Four observational groups were formed categorically, two groups of kinesiology students and two groups of elementary education students. Thirteen kinesiology students were originally involved in the data-collection procedures, six in one observational group and seven in another observational group. Unbalanced designs in generalizability analysis, however, introduce a number of computational concerns. Balanced designs are preferential (Brennan, 1983; Shavelson and Webb, 1981). Therefore, prior to a statistical analysis of the data, the size of each 129 observational group was reduced to five through a process of random deletion. Because females predominantly are enrolled in an elementary education certification program and because of the possibility of gender bias, all observers for this study were female. Human subjects approval for paid, adult volunteer subjects was obtained. Each observer was given a form letter explaining the purpose of the study and describing the rights and responsibilities of study participants. Informed written consent was obtained from each observer (see Appendix B). It should be noted that at the conclusion of the data-collection procedures the volunteers in one of the kinesiology groups refused payment for their services. Further discussion of this issue can be found in chapter 4. All observers were concurrently enrolled in undergraduate classes conducted by the study investigator at a major university in the Rocky Mountain Region. The elementary education observers were volunteers from two sections of an undergraduate class in "Physical Education and Health in the Elementary School". Coincident with their involvement in this project, these observers were studying elementary school physical education curriculum and methodology. The kinesiology observers were volunteers from an undergraduate class in "Human Development and Movement 130 Behavior" and were concurrently studying composite (total- body) developmental sequences for several fundamental motor skills. None of the observers, however, had previous training in either the composite sequencing or the component sequencing of hopping development, nor were they given this training in their respective courses of study. One limitation of this study concerns the method by which observers were assigned to observational groups. Random assignment did not take place. Because the data- collection procedure (to be described later in this chapter) required several evening Sessions with each observational group, the groups were formed according to the availability of the observers. The observational groups were described as follows: 1. Compositesgines -- Kinesiology majors rating the hopping performance of children using the Haubenstricker et al. (1975) total-body developmental sequence. 2. Composite E.E. -- Elementary education majors rating the hopping performance of children using the Haubenstricker et al. (1975) total-body developmental sequence. 3. Component Kines -- Kinesiology majors rating the hopping of children using the Halverson and 131 Williams (1985) arm and leg developmental sequences. ngponent E.E. -- Elementary education majors rating the hopping performance of children using the Halverson and Williams (1985) arm and leg developmental sequences. Procedures The data collection procedures will be described according to the following six steps: 1. 4. 5. videotaping the subjects performing the hopping task: editing the videotape: establishing the reliability and objectivity of the investigator: meeting with the observers: obtaining pre-training ratings of the data- collection videotape: conducting training sessions and obtaining post- training ratings of the data-collection videotape. 132 Videotsping the Spbjects Performing the Hopping Task The subjects were videotaped hopping a distance of 15 feet on their preferred foot. The 15-foot distance was chosen based on results from previous investigations. Jenkins (1930) and Keogh (1965) found that a large percentage of 5-year-old children are unable to complete a 50-foot hopping task without losing their balance (18-38%). Touwen (1979) found that 4- and 5-year-old children are capable of performing only five to eight hops consecutively. When validating the arm and leg sequences for hopping, Halverson and Williams (1985) used a 12-foot distance with 2- to 6-year-old children. The Michigan Education Assessment Program (MEAP, 1984) uses a distance of 30-feet to assess the hopping performance of fourth graders. Based on this research, the personal experiences of the investigator in rating the hopping performance of children 3 to 6 years of age, and the age level of the children in the study, a distance of 15 feet was considered most appropriate for this hopping task. Ulrich, Ulrich, and Branta (1988) found that hopping could be reliably assessed by one observer using the Haubenstricker et al. (1975) developmental sequence within five performance trials. Consequently, the subjects were given one warm-up trial and were then videotaped as they 133 completed the hopping task over five independent trials. Because the purpose of this study was not concerned with comparing the laterality of children's hopping performance, the children were asked to perform all five trials on their favorite foot. Occasionally, a child would begin hopping on the wrong foot or would stop hopping in the middle of a trial. If this occurred, the trial was aborted and a new trial was videotaped. A Curtis Mathis Camcorder, model BV800, was used to videotape the children. The subjects were videotaped from a side view in order to provide the best single viewing angle for assessing hopping performance (Haywood, 1986: MEAP. 1984: Roberton & Halverson, 1984). It is acknowledged by the investigator that two angles are preferable when analyzing hopping performance for the purpose of studying movement patterns. . The video camera was located perpendicular to the middle of the 15-foot hopping path and at a distance that allowed two additional feet to be viewed on either end of the path. The children were encouraged to begin and end hopping outside of the viewing range of the video camera. To facilitate the children's comprehension of the distance they were to hop, pylons were placed at both ends of the path and five feet beyond the ends of the path. Black, plastic tape was used to mark off one-foot intervals on the 134 path. The floor plan of the videotaping setup is depicted in Figure 4. Prior to the warm-up trial, all children were informed of the purpose of the study, given a verbal description and visual demonstration of the hopping task, and asked if they had any questions. All children were also told they could withdraw from participation in the study at any time during the videotaping session. The five trials for any one child were videotaped consecutively. Children were encouraged to "hop their very best all the way to the end." All children were given positive feedback about their performance. Mgiting the Videotspe Sixteen subjects were eliminated from the original pool of videotaped children for reasons previously identified. The remaining 59 subjects were rated by the study investigator in accordance with all three developmental sequences employed in the study (arms, legs, and total- body). Subsequently, the videotape was edited to produce the two videotapes used in the study. These videotapes included a data-collection videotape and a training videotape containing six distinct segments. The data-collection viQeotaps. From the 59 eligible subjects in the videotape pool, the performances of twenty 135 mcowmmom ocflomuoeo9> Lou swam molo .9 093099 D - €52 : _ . _ _ _ . . _ . . . _ _ . _ W— 136 children were randomly selected for actual assessment by the observers. A data-collection videotape was dubbed from the original videotape using these children as subjects. The training videotape. Using selected performances from those children not chosen for the data-collection videotape, six distinct training segments were dubbed onto a training videotape from the original videotape. Each segment was dubbed to contain hopping performances representative of a particular developmental sequence, thereby producing two video-taped training segments for each of the three developmental sequences. For each sequence. the first training segment depicted four to six classic examples of each step (stage) in the sequence and was shown during the first hour of observer training. The second segment for each sequence began with an abbreviated review of these classic performances, and then provided examples of performances in transition between the conventional step (stage) categorizations. This second segment was shown during the second hour of observer training. Establishingfithe Reliability43nd Objectivity of the Investigator The investigator of this study was responsible for training the observers in categorizing the hopping 137 performance of children. Therefore, it was pertinent that the investigator's skill in using the categorical definitions of the Halverson and Williams (1985) arm and leg sequences and the Haubenstricker et al. (1975) total-body sequence be determined prior to the training sessions. To facilitate this analysis, one of the original sequence investigators for each assessment technique was asked to rate the data-collection videotape. Measures of reliability and objectivity were computed for the study investigator and the sequence investigator. In order to determine the consistency of measurement between the study investigator and the sequence developers. three 20x2x5 (subjects by observers by trials) generalizability analyses were conducted -- one for each developmental sequence employed in the study. Cardinet et al. (1976, p. 129) indicated that a generalizability coefficient of .80 is acceptable for establishing observer generalizability. Frick and Semmel (1978) have proposed that a criterion- related agreement measure is more useful than a reliability measure when decisions about the objectivity of an individual's observational skills are to be made. Consequently, the percent of exact agreement between the study investigator and the sequence investigators was also calculated for each of the developmental sequences. Based 138 upon previous studies in motor development research, a criterion of 80% agreement was considered an acceptable level of objectivity (Langendorfer, 1987b: Roberton, 1977b, 1978a: Roberton & Langendorfer, 1980). Intra-observer reliability and intra-observer agreement for the study investigator were also examined. The investigator rated the data-collection videotape on two separate occasions. These occasions were one month apart. Measures of intra-observer generalizability and intra- observer percent agreement were calculated. Meeting with the Observers The first meeting with each observational group lasted no more than twenty minutes. During this time the following events took place: 1. the purpose of the study was explained; 2. a brief introduction to the concepts behind developmental sequences took place: 3. the written description(s) of the developmental sequence(s) appropriate to the observational group was/were distributed 4. the written description of each developmental sequence was read and any unfamiliar terms were 139 defined, but no further explanation of the description was offered. Obtaining Pre-training Ratings of the Data-Collection Videotape Less than one week after the initial meeting with each observational group, the observers were asked to categorize the hopping performance of the twenty subjects recorded on the data-collection videotape. On all occasions the four observational groups met independent of one another. Each observational group employing the Haubenstricker et al. (1975) developmental sequence met on one occasion for approximately 1 1/4 to 1 1/2 hours to complete pre-training rating of the data-collection videotape. The observational groups using the Halverson and Williams (1985) arm and leg developmental sequences required two observational sessions. one session for rating the arm action and one session for rating the leg action. Both sessions were approximately 1 1/4 to 1 1/2 hours. Robinson (1974) had found that rater fatigue occurred during 2-hour rating sessions. D. Ulrich (personal communication, April 8, 1988) has indicated that rating sessions should last no longer than 1 1/2 hours. Therefore the rating sessions for this study were limited to 1 1/2 140 hours and entailed the observation of 100 performance trials (20 subjects each performing 5 trials). Three 5-minute rest periods were given during each rating session. These occurred after 25, 50, and 75 trials had been categorized. Copies of the forms that were used by the observers to record their ratings can be found in Appendix C. To familiarize the observers with the content of the videotape and the time constraints of the rating sessions, each pre-training rating session began with a discussion of the procedures to be followed during the session. Three recordings of the data—collection videotape were simulated. one for each developmental sequence, and shown to the observers. These recordings consisted of two trials at each step (stage) in the development sequence. The rating of each performance on these recordings was not discussed with the observers, but this preliminary viewing did give the observers a chance to see examples of all the steps (stages) in the sequence they would be using to rate the data- collection videotape. It also allowed the observers to experience the time structure within and between each performance on the videotape. 141 Conducting Training Sessions and Obtaining Post-training Ratings of the Data-Collection Videotape All observers were trained to categorize the hopping performance of children using the developmental sequence(s) that corresponded to the observational groups to which they had been assigned. Several studies have shown that a one- hour training session in the assessment of gross motor performance is inadequate (Mielke & Chapman, 1987; Ulrich,Riggen, Ozmun, Screws, & Cleland, in press: Ulrich et al., 1988). Two l-hour training sessions were therefore conducted for each observational group. Specifically, two hours were allotted for training on the total-body sequence and two hours were allotted for concurrent training on both the arm sequence and the leg sequence. The first hour was devoted to the following points: 1. a verbal description and physical demonstration of the levels or stages in the developmental sequence(s) being studied; 2. the viewing of the first training segment on the training videotape that depicted classic examples of hopping performances at each level of development in the sequence(s). The second hour of training took place no more than five days after the first hour of training. All observers 142 were asked to review their written material prior to the second hour of training. The session began with a verbal and visual review of the levels in each developmental sequence. The second segment of the training videotape was then viewed. After each performance on this videotape, the observers were asked to verbalize their rating of the performance. A brief discussion on the rationale for their ratings occurred and the performance was viewed a second or third time if questions arose regarding the performance rating. The post-training rating sessions were held within a week following the final training session. Viewing of the simulated recordings of the data-collection videotape did not take place, otherwise the procedures for these sessions were identical to the pre-training rating sessions. The observational groups employing the Haubenstricker et al. sequence again met only once, whereas the observational groups using the Halverson and Williams arm and leg sequences required two rating sessions. Measurement Design The current study was conducted according to the framework of generalizability theory. Two G-study designs and twenty-five D-study designs were employed in several 143 statistical analyses. These designs, along with a discussion of the assumptions of generalizability theory, are presented in the following sections. Assumptions of Generalizability Theory Kane and Brennan (1979) have summarized the assumptions inherent in the generalizability models used in this study. These assumptions are typical general linear model assumptions: (a) all effects in the models are assumed to be sampled independently, and (b) the expected value of each effect over the population of persons and the universe of items is zero. Kane and Brennan also state: "It is particularly important to note that generalizability theory per se requires no assumptions about the form of the distribution of observed scores or score effects." (p. 36) Godbout and Schutz (1983) have noted the assumptions with regard to the measurement error of a generalizability analysis for a subjects x observers x trials design. Subject error. 1. Between trial variation in subject performance above or below a "universe" level of ability is referred to as subject error. 2. Systematic subject variation, that which may be 144 due to a learning or fatigue effect, is reflected in the variance component for the trial facet. Random subject fluctuation is accounted for in the variance component for the interaction of subjects and trials. A systematic interaction between subjects and trials is not assumed. Observer error. 1. 4. Systematic observer bias in using a rating scale, such as consistently overrating or underrating subjects' performances, is accounted for in the variance component for the observer facet. A systematic interaction between observers and subjects might be caused by a halo effect or an age or gender bias and would be evidenced in the variance component for subject x observer interaction. Random observer error is reflected in the variance component for the subject x observer x trial interaction. A systematic interaction between observers and trials is not assumed. 145 G- and D— Study Designs and Statistical_§nalyses The research designs for this study were designated according to the framework of generalizability theory as discussed by Brennan (1983). The use of GENOVA, a FORTRAN IV ggfleralized Analysis Of VAriance for balanced designs (Crick and Brennan, 1983), allows the four-phase procedure presented by Cardinet et al. (1981) to be condensed into C- study designs and D-study designs. Two G-study designs were employed in this project. G Study -- Design 1. The first G-study design was intended to examine the sources of measurement variance in the pre-training and the post-training data for all observers using the same developmental sequence to rate the hopping performances on the data-collection videotape. This design employed a 20x5x(5:2) structure and incorporated 20 subjects, 5 trials, and 5 observers nested within the 2 categories of elementary education and kinesiology majors. The subjects, trials, and observers were designated as random facets and the observer group was designated as a fixed facet. Variance components were computed according to a nested and mixed effects model. A Venn diagram and linear model for this design can be seen in Figure 5. One GENOVA was conducted before and one GENOVA was conducted after observer training for each of the three 146 .ouov x hpsum 0 wow Emuuwwn c=o> Hun“ "H nuance can Honor umocwq "m 0u50wh qoxwxuxa. and ‘0......'...' 00. o \ o o o :.I......*‘ 01. 8 '0 .0 ..°’Ooooo Ad: .3: 03.1 +0.6: o 03.1 ((9.01 H881 heuakv 9 New: . A: .01 .v: ...: o 1031. 8a-31-8a-§.3. 19¢: . at: .81 -3: use a. . l8: . A: .9. -9: -...an . “.31 9 m1 931 I961 Runaway o (dfito A 901 In... Idfilvo )8: o A: of. -..: (or: . (of . A a -317 .m:. .a-ua.. .ux. .a-ua.. In: . A: ... ...: . 1 "flaw 1 u 147 developmental sequences used to categorize the children's hopping performance (arm, leg, and composite). These computations resulted in the following six GENOVAs: 1. Pre-training GENOVA with the arm sequence as the dependent variable. 2. Post-training GENOVA with the arm sequence as the dependent variable. 3. Pre-training GENOVA with the leg sequence as the dependent variable. 4. Post-training GENOVA with the leg sequence as the dependent variable. 5. Pre-training GENOVA with the composite sequence as the dependent variable. 6. Post-training GENOVA with the composite sequence as the dependent variable. For each set of pre- and post- training GENOVAs, the magnitude of the variance components was examined to determine the effect of training observers in the assessment of hopping performance. These results are discussed in the next chapter. G Study -- Design 2. In order to generate variance components and generalizability coefficients for each category of observers using a designated developmental sequence, it was necessary to calculate several GENOVAs according to a second observational design. A 20x5x5 148 (subjects x observers x trials) design was employed during this phase of the study. All facets of this design were fully crossed and variance components were computed under a random effects model. Figure 6 provides a Venn diagram and linear model for this design. Twelve analyses were conducted, constituting pre- and post- training GENOVAs for each of the following observational conditions: 1. Arms Kines: Kinesiology students using the Halverson and Williams (1985) arm sequence: 2. Legs Kines: Kinesiology students using the Halverson and Williams (1985) leg sequence: 3. Composite Kines: Kinesiology students using the Haubenstricker et al. (1975) total-body sequence: 4. Arms E.E.: Elementary education students using the Halverson and Williams (1985) arm sequence: 5. Legs E.E.: Elementary education students using the Halverson and Williams (1985) leg sequence 6. Composite E.E.: Elementary education students using the Haubenstricker et al. (1975) total—body sequence: For each observational condition, the magnitude of the variance components was examined in order to determine the generalizability of ratings assigned before and after training. Additionally, comparisons of the various sources of measurement error in the post-training ratings were made 149 u x o x m "N cowmmo hpsum 0 wow Emuumwa sco> paw Hopoz wows“; "w casuah (08.: + IM$a+ (2.1 + a: (v1 + (.1 + 131+ e: .I g: -aa I. ngv + o‘otuooooooooooooooooo A1+u1IJuemno a usuunno mucosa huwawnswwdswoseo cw muoeuuo ooxfiu sou usesonaou 169 oussawm> a van .oususueuan kuuuawueua cw awou uausupsso n aha "ouoz mo.oa omeom.o mu.mn mvamn.o woman e>wuuaeu oo.ooa nmmnemd.a oo.ooa bravmon.a mam nasuoa .e.u"BOm. Hm.vn mahoahn.o mn.m« vaaanvn.o moo Heavamox ha.o moaanoo.o Hm.m meoowno.o or Bum oo.o mnmvooo.ol oo.o vnvuaoo.ou an Unao oo.o mmnaooo.ou va.o firemaco.o v Bu mN.H enumvao.o mv.m oomvuno.o we Hm vn.ma mvmmaam.o hm.eu nmmmmam.o «ma qum Hm.N mmaaomo.o mv.n revamvo.o ma um m«.o mnmaaoo.o na.o mmmnnoo.o v .8. mavens >m.o oomaano.o om.o bambooo.o m .Uuo. usuunno oo.o zmo mommmoo.ou mm.v zmo momnhmo.o H .0. huooeueu av.Ho manhmob.o m«.Hn «Honmov.o ma .mv auoonnsm mmmmwmmw mwmmwwmw mmmmdmmw musewunu mmmmmmm wommwus> Hobos & moseaws> Hmuoe & musedus> neewooa no 00u90m oustumom nuanumom Dualmum chateau Amososvom and. Mafiafinmwwasuesoo andsasua lunch one team new ousewum> useuuom can nousawumm eonswus> "Ha wanes 170 It is also interesting to examine the changes in error variance found in the nested observer facet (4.668 to 0.978), and in the observer category (4.388 to 08) facet. Of particular interest in these analyses is the reduction of error in the observer category facet. Although only 4.388 of the total variance in the pre-training results is attributable to this facet, this variance was reduced to 08 in the post-training results. .It would appear that prior to training there were some systematic differences between elementary education students and kinesiology students in using the arm sequence to rate hopping performances, and that these differences disappear with training. A further examination of this possibility is discussed later in this chapter. The Effects of Training on Using the Lsg Sequence to Assess Hopping Ability The estimated variance components and the percent of total variance for each facet in the pre-training and post- training analyses of the leg sequence are depicted in Table 12. Although not as dramatic as the results of the arm sequence, some effect following training in the use of the leg sequence to rate hopping performances is evidenced by pre- and post- training differences in the contribution of mmwuooousu swsuws peanez mue>wwnno u usuumno mucosa hudafinswanewesoo aw auoeuuo voxflu you uneconaoo 171 musswws> a pan .oususuouaa Hsowuanusua cw su0u causupssc n aha "ouoz m>.na oomno.o ma.oa omHnH.e woman 0>wusnex oo.ooa cahomoo.o oo.oon mnvnoam.o mam adsuoa .0.Uu90m. HN.MH muonomo.o mn.ma whambma.o moo Hssuwuex oo.o «mammoo.OI oo.o «mmnmoo.ol we Bum aa.o vhmmaoo.o mH.o vumnaoo.o no Uuao oo.o mndmooo.on mH.o «momuoo.o e Bu mo.N mammuao.o vu.~ meonmao.o me am ma.oa «whbdmo.o mv.m varomwo.o and qum ma.o . «endfloo.o mo.o omhvooo.o ma Um oo.o «Homooo.OI no.0 vornooo.o c .9. nasfiua mH.v unnamao.o N>.m mhbmovo.o m .uuo. umuuano c«.H zmo mmmeeoo.c om.H zao ommomao.o H .u. suoomuuo mo.am umor«av.o mv.«m wwmooam.o ma Am. auoonnsm wmmmwmmw mummwumm musmwum> mmmmwmmm .mmmmmmm musswws> H0909 a museuus> kuoa & musswus> monsoon no oUHDOm uuBIuaom uualumom ouhumum useless Amusosvem 004. aufinwnewwfluuosou newsflash nuaom use (mum wow menswus> acmouom one mousfiwumu musswus> "me wanna 172 variance components to total variance. The significance of these small shifts in variance contributions is not testable, but the notable differences that can be seen are as follows: 1. The percent of total variance attributable to subjects increased from 62.488 to 68.868. 2. The percent of total variance attributable to random observer error was reduced from 19.358 to 13.218. 3. The percent of total variance attributable to the nested observer error was reduced from 5.728 to 4.158. 4. Relative error variance was reduced from 16.158 to 13.758. There was a slight increase (< 28) in the variance due to the interaction between subjects and nested observers. There is no concrete information to explain its existence. The Effects of Training on Using the Comppsite Squence to Assess Hopping Ability The results of the pre-training and post-training GENOVAs for the composite sequence are summarized in Table 13. A slightly larger training effect was apparent in the newuoomueu swap“: veummz aum>ummao u ueoumno huoenu hufiflanewwasuesoo cw mucouuo oexwu uOu uneconEou 173 masseuse a van .oususweu«H Hsuwumwusum s“ Esau uausupeso u tho “ouoz ma.mn monmd.o vo.v« mhumn.o momma oewusfleu oc.ood mvonnam.o oo.oon medaaoo.o mam masuoa .o.uu90m. cm.ha bnbvuva.o Hm.bn oooonnm.o moo assowuou oo.o mammooo.on mm.o nmoeuoo.o we Bum «H.o naomooo.o on.o omnmaoo.o an UuBO oo.o vnvoooo.on oo.o nonoaoo.on e Bu mm.a ennvnao.o mm.o mmnmhoo.o up an em.ma mambmnd.o vn.>a oooomnd.o and qum ov.o unannoo.o oo.o enhvmoc.on ma um mo.o mmmvooo.o oo.o nmmnaoo.ol v AB. mfiowuh mm.v mmnmhmo.o oa.m omhmmao.o m Auuov usuuano oo.o zmo mocvmoo.ou mm.o 2&0 bmmmvoo.o H .0. huooeuuu vb.mm neroerv.o Ho.om mnemoo¢.o ma Am. uuoenasm mmmmmmmw mummwwmw mmmmflmmw mmmawwmm ammmwmw ousmwus> Hence & musewus> Heuoa & musswus> noouoea uo muwsom uualumom onslumom ouauoum ouaueum Amososuem euwnonaouv auwaanswwasuoseu unwswoua lumom use team may masseus> unmouem use mouwawunm masseus> "ma oases 174 composite sequence than in the leg sequence. The variance due to subjects increased from 50.018 to 58.748. Although little change is seen in the amount of total variance associated with either the nested observer facet (2.108 to 4.668) or the subject/nested observer interaction (17.348 to 16.848), the variance due to random observer error was reduced from 27.818 to 17.548. The overall reduction in relative error was from 24.048 to 18.858. Further confirmation of the study hypothesis suggesting there will be positive effects due to training is suggested by these data. As previously indicated, the significance of the pre- to post-training changes in the percent of variance attributable to error and to subjects is not testable. It appears, however, that training observers to use a composite sequence to rate hopping performances will reduce error variance. Discussion The analysis of pre- and post- training ratings for all observers suggests that there were less training effects evidenced in the ability of the observers to use the leg sequence than in the ability of the observers to use the arm or composite sequences. It is noted that the amount of total variance due to subject variation in the pre-training 175 data associated with the leg sequence was relatively high before training (62.488). This is especially true in comparison to the subject variation associated with the pre- training results of the arm sequence (31.288) and the composite sequence (51.018). Additionally, post-training subject variation for both the arm sequence (61.488) and the composite sequence (58.748) was still lower than the pre- training subject variation for the leg sequence. Observer experience with rating the data-collection videotape could account for the results obtained on the leg sequence data. Both the elementary education observers and the kinesiology observers using the arm and leg sequences to rate the hopping performances on the data-collection videotape required two viewings of the videotape in order to apply both sequences. For both categories of obseryers, the arm sequence was the assessment system adopted in the first rating session. Familiarity with the rating procedure and the videotape may have contributed to more reliable scoring during the second rating session when the leg sequence was employed. This interpretation does not account for the post-training results, however. The ease with which the written descriptions of the developmental sequences for hopping can be understood and employed by an untrained observer may provide a second explanation for the results obtained on the leg sequence 176 data. Immediately following all pre-training rating sessions the observers were asked if there were any parts of the categorical descriptions that were confusing and that may have affected their use of a sequence to rate the performances on the videotape. There was only one confusing concept in the Halverson and Williams (1985) leg sequence that might have affected the untrained observers' perceptions. This concept dealt with the terms "extension" and "pre-takeoff extension." The observers were unsure what was meant by these two terms. The observers expressed several concerns about the terminology of the arm sequence. These included (a) uncertainty about the position of the arms, (b) confusion with regard to bilateral arm "inactivity" that is further described as "usually slight and not consistent” (step 1). and (c) new and undefined terminology (especially for the elementary education students) such as "medially rotated", "winging movement", and "line of the trunk." The untrained observers employing the Haubenstricker et al. (1975) total-body sequence also expressed specific concerns about the categorical descriptions in the developmental sequence for the total body. Confounding terminology (again, especially for the elementary education students) included "surface" and "force production" in stage 1, "flexing the hip joint" and "bilateral manner" in stage 177 2, "vertical (frontal) plane" and "force production phase" in stage 3, and a lack of discussion on arm movement in stage 4. Both the elementary education observers and the kinesiology observers felt the composite descriptions were "too wordy” and difficult to interpret. The reaction of the untrained observers to the developmental sequence descriptions suggests that the leg sequence description was easier to interpret than the arm and total-body descriptions. This could account for the differences in the pre-training use of the three developmental sequences. It does not seem like a plausible explanation, however, for the post-training results. Certainly the one implication that can be suggested from the comments of the observers is the need to reword the developmental sequences for the arms and the body so that they are more easily interpreted by the practitioner. One final interpretation arising from the differences derived among the three sequences lies in the observational focus of the observer. It may be that the easier movements to see in the overall act of hopping are those of the legs. Hopping is a locomotor skill and, as such, might cause an observer to focus on that component of the body, the legs, that actually transports the individual through time and space. It also is possible that the arm movements are simply more difficult to perceive and evaluate than the leg 178 movements. The movements of the arms may (a) be more subtle than those of the legs, (b) be more erratic as they act and react to maintain balance and assist with force production, or (c) have greater degrees of freedom and range of motion, including that resulting from upper body rotation. A frontal view, in addition to the lateral view employed in this study, might assist the observer in more accurately evaluating the arm action. Current research by Petrakis (1986, 1987) on visual search patterns has examined the scanning and focusing patterns of individuals observing a movement performance. This line of research, coupled with generalizability analyses of developmental observational systems, may provide some clues on the types of experiences needed by undergraduate students that will assist them in developing expertise in identifying the critical elements of a developing motor skill. Analysis of the Developmental Sequences By Observational Groups Four 20x5x5 (subject by observers by trials) GENOVAs were conducted on the raw data for each of the three developmental sequences employed in this study. For each sequence these GENOVAs were administered on the (a) pre- 179 training ratings given by the elementary education observers, (b) post-training ratings given by the elementary education observers, (c) pre-training ratings given by the kinesiology observers, and (d) post-training ratings given by the kinesiology observers. Consequently, twelve GENOVA runs were administered. The GENOVA outputs provided G study analysis of variance tables (see Appendix E) and tables of model variance components, as well as decision (D) study variance components and generalizability coefficients. Twelve negative variance components were obtained in the G- study analyses. They were associated with either the trial facet or the observer/trial interaction in all cases but one. Consistent with previous manipulation of these components, all negative variance derivations were set to zero and GENOVA algorithm computations were employed. D-Study Optimization anq Classifications of Generalizability Coefficients The degree to which subjects' scores can be generalized to any set of observers and any set of trials is an important one in determining the usefulness of a developmental sequence as an observational assessment system. This degree is depicted in the subject score generalizability (G) coefficient, with .80 considered to be 180 an acceptable level of generalizability in this study. Given different combinations of measurement conditions, the degree of generalizability will be affected. When the number of observers and/or the number of trials constituting the measurement conditions are increased, the variance estimates associated with observers or trials can be reduced proportionally. This reduction has the effect of increasing the coefficient of generalizability and thereby strengthening the generalizability of the results. In order to obtain subject score generalizability coefficients for the pre-training and the post-training ratings, twenty-five decision (D) studies were conducted on each of the twelve GENOVA runs. The D studies were based on the various combinations of measurement conditions that could be derived from permutations of five trials and five observers. The GENOVA program recalculated the G-study variance components according to the D-study designs and computed the subject score generalizability coefficients from these calculations. Due to the limitation of the GENOVA program to differentiate on only a single facet, one hundred manual computations additionally were administered on the data for each sequence. These computations consisted of four sets of 25 post-training D studies. Two of the four computational sets derived inter-observer coefficients of generalizability 181 for the elementary education observers and for the kinesiology observers. The other two computational sets derived inter-trial coefficients of generalizability for each group. Inter-observer generalizability addresses the consistency with which observers are able to differentiate subject performance on a set of performance trials, thereby indicating the degree to which results can be generalized across different observers. According to the formula set forth by Godbout and Schutz (1983), the subjects and the set of trials are random facets and serve as the facets of differentiation (or the objects of measurement). Observers, also a random facet, serve as the facet of generalization (or instrumentation). Inter-trial generalizability is concerned with the generalizability of subjects' scores to other sets of trials. The presence of observer error in an inter-trial generalizability formula appears to bias any interpretation of the G coefficient as a measure of trial stability. When observers are considered a random facet, however, the inter- trial G coefficient does express the ability to generalize subjects' scores over trials given any set of observers. Godbout and Schutz (1983) indicate that when observer error is "perfectly" absent, inter-trial generalizability can be considered a measure of stability over trials. 182 The inter-observer and inter-trial computations for this study were completed by the study investigator according to the formulas furnished by Godbout and Schutz (1983). The G coefficients obtained are presented in tables in the forthcoming sections of this chapter. The results for each developmental sequence will be addressed independently, followed by a comparison of the analyses for the three sequences and an interpretive discussion. Arm Segpence Analysis Tables 14 and 15 display the variance estimates and the percent of total variance derived for each facet in the pre- training and the post-training analyses conducted on the arm sequence data. The results provided further evidence of the positive effects of training on the ability of observers to use the Halverson and Williams (1985) developmental sequence for arm action in assessing the hopping performances of children and provided further confirmation of the research hypothesis that addresses this issue. For both categories of observers these effects were confirmed in the following relevant variance adjustments: 183 nn.ov ammom.o Ho.mm orenm.o uouuu e>fiumwo¢ oo.ooa wanmhmm.d oo.ooa vomammm.o mmv maauoa on.bd oomNmHN.o am.on ammmmon.o eon .0.80m. Huspflnez oc.o ooomooo.ou bn.o vamonoo.c 0H 90 ou.H amoOHao.o vn.m mamnnmo.o me am bn.dn Hammmm«.o mv.wm oombvou.c or On Ho.o «emaooo.o vm.o vmonmoo.o v .9. namwua mn.a mhmmhao.o ov.H oooovao.o v on awo>uemao hn.mm manomn>.o mw.vn anamwvn.o ma Am. nuoennsm musswum> ouwaflumm ousmfiumw mumfiwumu mmmwmuh oUGflWus> Hench & moswfiHs> kuoa & masseus> awesome «0 oousom ouenumom uualumom oualmum oueloum Amusesvom and. coauwospu humusmewam wow mosmwum> usmuuom pas neueauumu oussfius> muo>uemno nod eHnt 184 ob.mn mbmoN.o mH.om bdmmb.o HOHHH m>wuma0u oc.coa nmomboo.u oc.coa mmmowdv.d mac edmuoa ov.ud abomvaa.o om.mu Hahmmhn.o «on Am.BOmv Huspwmom oo.o meovooo.o: oo.o Hardwoo.on 0H 90 oa.a HNmOHHo.o om.n mmnmmvo.o we Hm mN.ma nwbhnma.o oo.mn mommmmn.o or Om vn.o abovnoo.o oo.o mmnooco.cl c As. adswua bv.o enuhvoo.o mm.b mmnmeou.o v .03 nuw>uomno nv.0h vhmmmo>.o ma.mn mmmnnam.o ma Am. auooflnzm mmmmwmmm .mwmmwwmw wmmmwmmw mwmawwmw mmmmwmu ousmfiua> Hosea & oossflus> asuoa & ousswus> monsoon uo euusom unanunom uuBIumom owslmum oualmum Amusosqmm Eu usouwom use mousfiwumm eucswum> “ma oHnma 185 1. Subject Variability a. Increased from 34.888 to 58.378 for the elementary education observers. b. Increased from 36.258 to 70.438 for the kinesiology observers. 2. Random Observer Error a. Reduced from 30.988 to 17.308 for the elementary education observers. b. Reduced from 26.608 to 12.408 for the kinesiology observers. 3. Subject/Observer Interaction a. Reduced from 26.498 to 21.278 for the elementary education observers. b. Reduced from 26.068 to 15.268 for the kinesiology observers. 4. Relative Error Variance a. Reduced from 62.818 to 40.238 for the elementary education observers. b. Reduced from 56.158 to 28.768 for the kinesiology observers. The post-training results for both categories of observers reveal a relatively large percent of total variance attributable to the subject/observer interaction. This variation reflects a difference in the ratings given to subjects by the observers and suggests that even after two hours of training some degree of observer disagreement remains. A comparison of the pre-training and the post-training G coefficients for subject score generalizability also serves to verify an improvement in the ability of the observers, as a result of training, to differentiate the arm 186 movements of children when they are hopping (see Table 16). The results can be summarized here by examining the single observation coefficients of generalizability derived from the pre-training analyses and the post-training analyses. The G coefficient for the elementary education observers increased from .36 before training to .59 after training, while the G coefficient of the kinesiology majors increased from .39 before training to .71 after training. A comparison of the post-training results among observational groups also present some interesting interpretations. The percent of total variance attributable to subjects was smaller for the elementary education observers than it was for the kinesiology observers (58.378 compared to 70.438). Coupled with a relative error variance for the elementary education observers (40.238) that is larger than the relative error variance for the kinesiology observers (28.768), it would appear that the kinesiology observers are more consistent in their arm sequence ratings. This interpretation is substantiated by the post- training G coefficients found in Table 17. Specifically. subject score G coefficients indicate that two observers viewing three trials were necessary conditions for the elementary education observers to achieve a minimal acceptance level (.81) of generalizability, whereas two kinesiology observers viewing three trials obtained a 187 Table 16: Subject Score Generalizability Coefficients Pre- and Post- Training for All Observers (Arm Sequence) Measurement El. Ed. Kinesiology Conditions Observers Observers O T Pre Post Pre Post 1 1 .36 .59 .39 .71 1 2 .44 .65 .47 .76 1 3 .47 .68 .50 .78 1 4 .50 .69 .52 .79 1 5 _ .51 .70 .53 .80 2 1 .51 .74 .55 .83 2 2 .60 .79 .63 .86 2 3 .63 .81 .66 .87 2 4 .66 .82 .68 .88 2 5 .67 .82 .69 .89 3 1 ~.59 .80 .63 .87 3 2 .68 .84 .71 .90 3 3 .71 .86 .74 .91 3 4 .73 .87 .75 .92 3 5 .74 .87 .76 .92 4 1 .64 .84 .69 .90 4 2 .73 .88 .76 .92 4 3 .76 .89 .79 .93 4 4 .78 .90 .80 .94 4 5 .79 .90 .81 .94 5 1 .67 .86 .72 .91 5 2 .76 .90 .79 .94 5 3 .79 .91 .82 .94 5 4 .81 .91 .83 .95 5 5 .82 .92 .84 .95 Mote: O = Observers T 3 Trials El. Ed. - Elementary Education 188 Table 17: Subject Score, Inter-Observer, and Inter-Trial Generalizability Coefficients for All Observers (Arm Sequence) Measurement Subject Conditions §ggre Inter-Observer Inter-Trial O T EE Kine EE Kine EE Kine 1 1 .59 .71 .61 .72 .81 .86 1 2 .65 -.76 .66 .77 .90 .93 1 3 .68 .78 .69 .79 .93 .95 1 4 .69 .79 .70 .79 .94 .96 1 5 .70 .80 .70 .80 .96 .97 2 1 .74 .83 .76 .84 .87 .91 2 2 .79 .86 .80 .87 .93 .96 2 3 .81 .87 .81 .88 .95 .97 2 4 .82 .88 .82 .89 .96 .98 2 5 .82 .89 ' .83 .89 .97 .98 3 1 .80 .87 .82 .89 .90 .94 3 2 .84 .90 .86 .91 .95 .97 3 3 .86 .91 .87 .92 .96 .98 3 4 .87 .92 .87 .92 .97 .98 3 5 .87 .92 .88 .92 .98 .99 4 1 .84 .90 _ .86 .91 .91 .95' 4 2 .88 .92 .89 .93 .96 .97 4 3 .89 .93 .90 .94 .97 .98 4 4 .90 .94 .90 .94 .98 .99 4 5 .90 .94 .90 .94 .98 .99 5 1 .86 .91 .89 .93 .92 .95 5 2 .90 .94 .91 .94 .96 .98 5 3 .91 .94 .92 .95 .97 .98 5 4 .91 .94 .92 .95 .98 .99 5 5 .92 .94 .92 .95 .98 .99 Note: 0 - Observers T 8 Trials 33 - Elementary Education Students Kine I Kinesiology Students 189 G coefficient of .87. A G coefficient equal to .80 was obtained by the kinesiology observers when minimal conditions were one observer and five trials. These data do not support the hypothesis that equivalent training would result in little differences in the generalizability with which kinesiology students and with which elementary education students rate the hopping performance of children. Shavelson et al. (1989) have noted that if the variance component for a facet of instrumentation is small and its interaction with other facets also results in small variance components, then adding more observations to that facet will have less effect than adding observations to a facet of instrumentation with larger variance. The amount of variance in the arm sequence data that was due to trials and to trial interaction with other facets was minimal for both categories of observers, a fact attested to in the relatively high (>.81) inter-trial coefficients of generalizability (see Table 17). In contrast, that portion of the error variance attributable to the interaction of observers with other facets was substantial. Concordant with these results is the degree to which all the coefficients of generalizability increased when observers were added to the design as opposed to when trials were added to the design. For example, adding one observer to the design increased the subject score G coefficient from 190 .71 to .83 for kinesiology observers rating one trial, whereas adding a trial to the design only increased the coefficient to .76. Plausibly, optimizing the measurement conditions by increasing the number of observers presents problems of practical concern. Most physical education classes employ only one teacher and it is far easier to add viewing trials than it is to add observers to the classroom. It is important to summarize the results of the D studies by noting the minimal conditions of measurement that were required to reach an acceptable level of generalizability for subjects' scores. In accordance with the inter-observer results, the minimal conditions of measurement for the elementary education observers were two observers viewing two trials. For the kinesiology observers the minimal conditions were one observer viewing five trials. Leg Seguence Analysis The data from the pre- and post- training analyses discussed previously (observers nested within observer category) implied that both the elementary education observers and the kinesiology observers were more adept at rating the leg action in the hopping performances of children than at rating the arm action or the total body 191 action. Further credence is added to this interpretation when the data for the leg sequence are examined separately from other data. The variance estimates and the percent of total variance associated with each facet in these analyses are located in Tables 18 and 19. The variance due to subjects for both groups of observers lent more support to the positive effect of training observers to rate the leg action of hopping performances than did the pre- and post- training data computed with observers nested within categories. The amount of variance attributable to subjects increased from 56.218 to 62.608 for the elementary education observers and from 71.578 to 79.118 for the kinesiology observers. Interestingly, a slight increase was seen in the variance component for the subject/observer interaction in the data of both the elementary education observers (10.098 to 11.688) and the data of the kinesiology observers (6.998 to 8.598). These figures imply that observers were rating subjects differently both before and after training. Despite the increase in the subject/observer variance component, the proportion of total variance due to the remaining random and systematic observer errors (observer variance + subject/observer variance + residual variance) was reduced for both groups of observers following training. 192 COHUUUflUN %HM&C030HN HON 00C0flhfl> uflmuhflh 0C6 QUUOEflumN OUGGHHU> na.mn mvamd.o vo.mn annmu.o woman 0>Husaou oo.ooa mmnmmwm.o oo.ooa manahnm.o mac nasuoa «m.ma «Humnca.o m«.vu manonom.c «on .o.aOm. assessed b«.o maumaoo.o oo.o wnoaaoo.o1 as 80 nm.u «mammao.o mm.o mmwhmoo.o me am mm.na moahhbo.o mo.oa bmhvvwo.o we Om oo.o mmomnoo.on mn.o moomnoo.o v .9. easfiua on.h mannavo.o ov.m nohmobo.o v .0. awo>uoano om.Nm ooomwdv.o u«.om marmorv.o ma Amy mavennsm oussfiws> mueswumw MMdswumw ousawumm eopomwh oussfiuo> Hmuoa a ousswuu> Hmuoa a oosswue> monsoon no ouwsom analysed unaluaom oualoum owaloum Amososvum boa. nuo>uomno "ma OHQMB 193 vu.ou mmroa.o mw.¢N nmomfi.o Houum Obwumaom oo.ooa enhvnnm.o oo.ooa bnhvaob.o mas nausea nu.oa accommo.o mm.va wuomNHH.o eon .0.80m. Henpwmom m«.o mmomaoo.o mm.o vbvvvoo.o mu BO no.H Hanvmo0.o ma.n mowvvno.o me am mm.m Huvmmvo.o mm.o Harmnmo.o or Om oo.o aarmoco.on oo.o moaeooo.on v AB. nasdus mn.o whomaoo.o oo.n mahonuo.o e “O. auo>uomno «H.mh mamcnnv.o hm.ab nnaoomn.o ma Am» uuueflnsm mwmmwmmw .mwmmflmmw ounm4us> mummwwmw mmmmmmu Imwmmwmmw Hmuoe & oosswus> Heuoa & mossfiuu> awesome «0 wowsom oualuaom owalumom amalgam oualuwm Aeosesuom owns muo>wmmno hoonowmmswx MOM wuswwus> usooaom pas mousfifiumu mosswus> "ma Gangs 194 For the elementary education observers this reduction was from 42.758 to 34.608, while for the kinesiology observers this reduction was from 24.678 to 19.588. Error reduction can also be verified by examining the proportional changes in relative error variance. This statistic was reduced from 34.048 of total variance to 29.838 of total variance for the elementary education observers, and from 24.858 of total variance to 20.248 of total variance for the kinesiology observers. The increase in subject score G coefficients for both observer groups following training further supports the interpretations regarding training effects (see Table 20). The single observation coefficient of generalizability for the elementary education observers was increased from .62 to .68, while the single observation coefficient of generalizability for the kinesiology observers was increased from .74 to .80. Consistent with the arm sequence data, the variation in the leg sequence data that results from the trial facet is minimal. In all four analyses the trial, subject/trial, and observer/trial variance components account for less than 58 of the total variance. This relative consistency in subjects' scores from trial-to-trial is reflected in the highly acceptable inter-trial coefficients of generalizability (see Table 21). The single observation, 195 Table 20: Subject Score Generalizability Coefficients Pre- and Post- Training for All Observers (Leg Sequence) Measurement El. Ed. Kinesiology Conditions Observers Observers O T Pre Post Pre Post 1 1 .62 .68 .74 .80 1 2 .71 .75 .82 .85 1 3 .75 _ .78 .85 .86 1 4 .77 .79 .86 .87 1 5 .79 .80 .87 .88 2 1 .76 .79 .84 .88 2 2 .83 .85 .89 .91 2 3 .86 .87 .91 .93 2 4 .87 .88 .92 .93 2 5 .88 .89 .93 .93 3 1 .82 .84 .87 .91 3 2 .88 .89 .92 .94 3 3 .90 .91 .93 .95 3 4 .91 .91 .94 .95 3 5 .92 .92 .95 .95 4 1 .86 .87 .89 .93 4 2 .91 .91 .93 .95 4 3 .92 .93 .95 .96 4 4 .93 .93 .95 .96 4 5 .94 .94 .96 .96 5 1 .88 .89 .90 .94 5 2 .92 .92 .94 .96 5 3 .94 .94 .95 .97 5 4 .94 .94 .96 .97 5 5 .95 .95 .96 .97 Note: 0 - Observers T - Trials El. Ed.‘ - Elementary Education 196 Table 21: Subject Score, Inter-Observer, and Inter-Trial Generalizability Coefficients for All Observers (Leg Sequence) Measurement Subject Conditions Score Inter-Observer Inter-Trial O T EE Kine EE Kine EE Kine 1 1 .69 .80 .70 .80 .82 .88 1 2 .75 .85 .76 .85 .90 .94 1 3 .78 .86 .79 .87 .93 .96 1 4 .79 .87 .80 .88 .95 .97 1 5 .80 .88 .81 .88 .96 .98 2 1 .80 .88 .83 .89 .87 .93 2 2 .85 .91 .87 .92 .93 .96 2 3 .87 .93 .88 .93 .95 .97 2 4 .88 .93 .89 .93 .96 .98 2 5 .88 .93 .89 .94 .97 .98 3 1 .84 .91 .88 .92 .90 .95 3 2 .89 .94 .91 .94 .95 .97 3 3 .91 .95 .92 .95 .96 .98 3 4 .91 .95 .92 .95 .97 .99 3 5 .92 .95 .93 .96 .98 .99 4 1 .87 .93 .90 .94 .91 .95 4 2 .91 .95 .93 .96 .95 .98 4 3 .93 .96 .94 .96 .97 .98 4 4 .93 .96 .94 .97 .98 .99 4 5 ..94 .96 .94 .97 .98 .99 5 1 .89 .94 .92 .95 .92 .96 5 2 .92 .96 .94 .97 .96 .98 5 3 .94 .97 .95 .97 .97 .99 5 4 .94 .97 .95 .97 .98 .99 5 5 .95 .97 .96 .97 .98 .99 Note: 0 I Observers T I Trials EE - Elementary Education Students Kine a Kinesiology Students 197 inter-trial coefficients of generalizability for both the kinesiology observers and the elementary education observers were above the .80 acceptance level, with all but one of the remaining G coefficients at .90 or above. One implication from the low variance associated with the trial and the trial interaction components is that an improvement in subject score generalizability and inter- observer generalizability will best be achieved by increasing the number of observers in the measurement conditions. This was a situation that was evidenced in the arm sequence data as well. Because of the ability of the observers to apply the leg sequence categories to hopping performances more consistently than the arm sequence categories, the issue of how to change measurement conditions in order to improve generalizability is not as perplexing. The results of this series of D studies indicate that only one observer was required for both the elementary education observers and the kinesiology observers to achieve acceptable G coefficients (.80) in subject score and inter- observer generalizability. For the elementary education observers the minimal conditions of measurement for subject score generalizability were one observer rating five trials. For the kinesiology observers the minimal conditions of measurement were one observer rating one trial. Concordant 198 to the analyses on the arm sequence data, the leg sequence analyses suggests that the kinesiology observers are more consistent in rating the leg action of hopping performances than are the elementary education observers. The leg sequence data also invalidate the hypothesis suggesting that little differences would be seen in the ratings of the kinesiology students and the ratings of the elementary education students when the two groups of observers are given equivalent training. Composite Seggence Analysis The GENOVA analyses conducted on the composite sequence are summarized in the variance components and the percent of total variance figures presented in Tables 22 and 23. Analogous to all the data results presented thus far, an improvement in the consistency with which both categories of observers were able to rate the hopping performance of children from a composite perspective is reflected in the post-training variance components and the subject score generalizability coefficients. 199 mm.mn wnddn.o mo.mv mmmm¢.o Houuu 0>fiuofiom oo.ooa momvoom.o oo.ooa hummmom.o mmv masuoa H«.an Hwaan.o en.hm vama>v«.o «on .0.90m. Huspuaox vo.o mannooo.o mv.o oanavoo.o ca 80 om.N vhmmwmo.o mN.H vhquao.o me am pm.HH mmaowoa.o mN.H~ hnbmnma.o or Om no.0 mnemooo.o oo.o vbvvnoo.ol v .av madame mm.o «mammoo.o oo.o hmhmvoo.OI v A0. uuo>uoano wm.mm momebhm.o mm.mv mamaomv.o mu .m. muoonnsm mwmmwmmw mmmadmmw mwmmmmmw mmmmwwmm mmmmmmm Immmmwmmw Hobos x mosswuw> Huuoa a ousmwus> awesome uo wouSOm uuanumom owhluaom ouanmum oualmum Amososoom ouwnooaou. nue>uonno soaususpm aswusmaoau sou ousswsm> useowem one mousswuau masseus> “an canes 200 oo.hn menon.o bv.nc mumma.o woman ebfiuenem oo.ooa mhmauah.o oo.ooa nmmnmho.c mac mneuoa mm.md nmbmumo.o mH.mN mannema.o eon .e.BOm. Hesownoa Hm.o enhvaoo.o oo.o cameooo.OI Bo am.o cedonoo.o we.” oHnHuHo.o am nm.n~ «mombmd.o am.NH noaommo.o Om mo.o manwooo.o oo.o maonooo.ol .B. sneaks mm.m memmwoo.o ch.m enmeeno.o .o. mue>uenno mn.nm mpommrm.o mw.on Hmvnmvn.o .m. uuoennsm mwdewue> eueaflumu ousewwe> ouefiwunm eooeouh ousmwwe> Heuos a easeflue> Hence & ensewue> awesome uo eousom ouauueom onslumom oneness ousteum .eusesvem euwmonfiou. mue>uomno mue>uemno hooaofineswx HOH ensewue> unmouem one moueawunm eusewue> "mm OHQMB 201 The variance components of the elementary education observers' data suggest a sizeable improvement in scoring consistency following training. The amount of variance attributable to subjects increased from 49.658 to 63.568. Subject/observer error was reduced from 21.298 to 11.678, and random observer error was reduced from 27.348 to 21.218. The reduction in relative error variance was from 49.898 to 35.388. The improvement in the ability of the kinesiology observers to employ the composite sequencing technique was not as dramatic as that of the elementary education observers. Variance due to subjects increased slightly from 50.838 to 53.358. A sizeable and desirable reduction in the random error of these observers is reflected in the magnitude of the residual variance component (29.168 was reduced to 12.968). Relative error variance was reduced from 43.478 to 37.008. Once again, the research hypothesis proposing a reduction in measurement error for all observers as a result of training was partially substantiated by the data of both the kinesiology observers and the elementary education observers. It should be noted that despite an overall reduction in relative error variance, the variance attributable to subject/observer interaction increased from 12.528 to 23.538 following training. Previous discussions in this chapter 202 have examined the variance associated with the subject/observer interaction and found that this variation may be attributable to differences in the ratings given to subjects by observers. It may be that the subject/observer interaction is a reflection of the degree of observer disagreement. Certainly a relatively large percentage of the total variance in the post-training, composite data of the kinesiology observers was associated with the subject/observer interaction (23.538). This measure of observer error contributed considerably to relative error variance, thereby reducing the consistency of the post- training ratings for the kinesiology observers. Interestingly, the percent of total variance attributable to observers increased from 5.708 to 9.368. Taylor (1979) and Godbout and Schutz (1983) have identified the variance due to the observer facet as systematic observer error. Godbout and Schutz further suggest that this variance is due to observer bias and may be a reflection of the tendency for some observers to overrate or underrate the performance of all subjects consistently. This error may also be caused by an age bias in which some observers give higher ratings to older-looking subjects and lower ratings to younger-looking subjects. A third possibility for systematic observer error is gender bias. 203 Ulrich et al. (1988) suggested that training procedures in developmental assessments need to highlight these biases. The training procedures for this study were explicit with regard to these phenomena. It is difficult to speculate on the factors contributing to the increase in observer bias that occurred in the ratings of the kinesiology observers after training. The G coefficients for subject score, inter-observer, and inter-trial generalizability are presented in Table 25. In all four of the composite analyses, the error variance due to trials or due to the interaction of trials with any other single facet in the design was minimal (< 2.58). This is substantiated in the relatively high inter-trial coefficients of generalizability, just as it was with the arm sequence data and the leg sequence data. Despite the less extensive improvement in the observational abilities of the kinesiology students as opposed to the elementary education students, the post- training generalizability coefficients for both groups imply positive effects as a result of training the observers to assess the action of the total body in hopping performances. Single observation G coefficients for subject score generalizability increased from .54 to .59 for the kinesiology observers and from .50 to .64 for the elementary education observers (see Table 24). 204 Table 24: Subject Score Generalizability Coefficients Pre- and Post- Training for All Observers (Composite Sequence) Measurement El. Ed. Kinesiology Conditions Observers Observers O T Pre Post Pre Post 1 1 .50 .64 .54 .59 1 2 .58 .73 .64 .64 1 3 .62 .76 .69 .66 1 4 .64 .78 .72 .66 1 5 .65 .79 .73 .67 2 1 .66 .77 .69 .74 2 2 .73 .84 .78 .78 2 3 .76 .86 .81 .79 2 4 .78 .87 .83 .80 2 5 .78 .88 .84 .80 3 1 .74 .83 .76 .81 3 2 .80 .88 .84 .84 3 3 .82 .90 .86 .85 3 4 .84 .91 .88 .85 3 5 .84 .92 .89 .86 4 1 .79 .86 .81 .85 4 2 .84 .90 .87 .87 4 3‘ .86 .92 .89 .88 4 4 .87 .93 .90 .89 4 5 .88 .93 .91 .89 5 1 .82 .88 .83 .87 5 2 .87 .92 .89 .90 5 3 .88 .93 .91 .90 5 4 .90 .94 .92 .91 5 5 .90 .95 .93 .91 Note: 0 - Observers T - Trials El. Ed. - Elementary Education 205 Table 25: Subject Score, Inter-Observer, and Inter-Trial Generalizability Coefficients for All Observers (Composite Sequence) Measurement Subject Conditions Score Inter-Observer Inter-Trial O T EE Kine EE Kine EE Kine 1 1 .64 .59 .67 .60 .76 .86 1 2 .73 .64 .74 .64 .87 .93 1 3 .76 .66 .77 .66 .91 .95 1 4 .78 .66 .79 .67 .93 .96 1 5 .79 .67 .80 .67 .94 .97 2 1 .77 .74 .80 .75 .84 .91 2 2 .84 .78 .85 .78 .91 .95 2 3 .86 .79 .87 .79 .94 .97 2 4 .87 .80 .88 .80 .96 .98 2 5 .88 .80 .89 .80 .96 .98 3 1 .83 .81 .86 .82 .88 .93 3 2 .88 .84 .90 .84 .93 .96 3 3 .90 .85 .91 .85 .96 .98 3 4 .91 .85 .92 .86 .97 .99 3 5 .92 .86 .92 .86 .97 .99 4 1 .86 .85 .89 .85 .90 .94 4 2 .90 .87 .92 .88 .94 .97 4 3 .92 .88 .93 .88 .96 .98 4 4 .93 .89 ' .94 .89 .97 .99 4 5 .93 .89 .94 .89 .98 .99 5 1 .88 .87 .91 .88 .91 .95 5 2 .92 .90 .94 .90 .95 .97 5 3 .93 .90 .94 .91 .97 .98 5 4 .94 .91 .95 .91 .98 .99 5 5 .95 .91 .95 .91 .98 .99 Note: 0 I Observers T I Trials EE I Elementary Education Students Kine I Kinesiology Students 206 The elementary education observers were just short of reaching an acceptable (.80) level of subject score generalizability with one observer viewing five trials, G = .79. Two observers viewing two trials did result in a G- coefficient of .84. The minimal measurement conditions required for the kinesiology observers to reach acceptable (.80) subject score generalizability were not attained until ratings were averaged over two observers and four trials. As with the leg sequence data and the arm sequence data, the generalizability coefficients derived from the total-body ratings do not support the research hypothesis that equivalent training of the observational groups will result in little difference between the groups in the generalizability of their ratings. Nor do these results appear to support the findings of Ulrich et al. (1988) which suggest that hopping performance can be rated reliably (G = .88) with the Haubenstricker et al. (1975) total-body sequence by one observer viewing three trials. It should be noted that the observers in the Ulrich et al. (1988) study were graduate students in a motor development and an adapted physical education class. Although these observers had no previous training with using developmental sequences to assess fundamental motor skill performance, it may be that their level of overall training in physical education was greater than that of the undergraduate kinesiology students 207 in this study. A more complete profile of the observers in the Ulrich et al. study would be necessary in order to confirm this supposition. Additionally, personal communication with Beverly Ulrich (June 5, 1989) revealed that the hopping performances depicted on the Ulrich et al. data-collection videotape were specifically selected to represent all the stages of performance in the composite sequence. The random selection of subjects for the data-collection videotape employed in the current study may have increased the likelihood that hopping performances in transition between stages were depicted on the videotape. Such performances would create more variability in observer ratings and subsequently, lower generalizability. This is a supposition that is difficult to support without comparing the data-collection videotapes from the two studies. It appears that future research in the generalizability of rating motor skills is necessary and should take into account several factors: (a) the experience of the observers with motor skill assessment: (b) the random selection of subjects: and (c) the examination of observational generalizability in the field setting. 208 Comparison of the Analyses of the Three Develqpmental Seguences Data bearing on the generalizability of the subjects' scores over observers and trials are summarized in Tables 26 and 27 for each of the three developmental sequences and the two categories of observers. Table 26 summarizes the coefficients of generalizability obtained for the D studies conducted on 1, 2, and 3 observers viewing 1, 2, 3, 4, and 5 trials. Table 27 reports the percent of total variance attributable to each facet in the six post-training analyses. Coefficients of generalizability are useful in determining the minimal conditions of measurement that would be required for reliable scoring of children hopping. The leg sequence proves to be the most reliably scored sequence of the three developmental sequences examined in this study. Both elementary education students and kinesiology students achieved acceptable generalizability (.80) with a single observer. For the elementary education students five viewing trials were necessary and for the kinesiology observers one viewing trial was all that was necessary. 209 Table 26: Post-Training Generalizability Coefficients For All Observational Groups Measurement Arm Leg Composite Conditions Seguence Segpence Seguence O T EE Kine EE Kine EE Kine 1 1 .59 .71 .68 .80 .64 .59 1 2 .65 .76 .75 .85 .73 .64 1 3 .68 .78 .78 .86 .76 .66 1 4 .69 .79 .79 .87 .78 .66 1 5 .70 .80 .80 .88 .79 .67 2 1 .74 .83 .80 .88 .77 .74 2 2 .79 .86 .85 .91 .84 .78 2 3 .81 .87 .87 .93 .86 .79 2 4 .82 .88 .88 .93 .87 .80 2 5 .82 .89 .88 .93 .88 .80 3 1 .80 .87 .84 .91 .83 .81 3 2 .84 .90 .89 .94 .88 .84 3 3 .86 .91 .91 .95 .90 .85 3 4 .87 .92 .91 .95 .91 .85 3 5 .87 .92 .92 .95 .92 .86 Note: 0 I Observers T I Trials EE I Elementary Education Students Kine I Kinesiology Students . 210 00.hn vu.0« 05.0N mn.mn n0.mn nu.0¢ noun” Obduedom 00.NH 00.0H 00.NH HN.HN N0.MH on.bH .O.BOm. Hoavflumx HN.0 0N.0 00.0 v0.0 bN.0 00.0 80 mm.0 N0.H 0H.H 0m.N nm.« 00.H Em mm.nu mm.» om.mH hm.HH 00.HH hfl.du Om 00.0 00.0 00.0 no.0 00.0 H0.0 .8. maefiua mn.m mn.0 bv.0 00.0 0n.h mn.H .0. muo>uonao mn.nm HH.mb nv.0h om.n0 00.nm hm.0m .m. nuOOnnflm maou eueq mead maou mum; wand wowuefiwe> season eue>ueeno NuoHowNdeu mue>uenno .om .Hu memhaesd newsweualuaom HH< a“ eueoem Had 0» maneusnwuuud easewue> Hence uo useuuem "hm eanea 211 For both the elementary education students and the kinesiology students, acceptable generalizability levels for subjects' total-body scores required more extensive measurement conditions than the levels for subjects' leg scores. For the elementary education observers, a G coefficient of .84 was obtained when the ratings were averaged over two observers and two trials. A minimal G coefficient (.80) was not evident in the scores of the kinesiology students until ratings were averaged over two observers and four trials. Given that observers employing a composite sequence to rate hopping must view and analyze actions of the total-body as opposed to the actions of the arms alone or the legs alone, these results are logical. They partially support the research hypothesis that measurement conditions for the total-body sequence would be more extensive. The results obtained for the arm sequence ratings were mixed. The kinesiology observers required less extensive measurement conditions for the arm sequence than for the total-body sequence. A single observer viewing five trials met the minimal conditions of measurement (G =.80). The minimal conditions for acceptable generalizability of the elementary education observers using the arm sequence were slightly more extensive than the conditions required by this same category of observers for the total-body sequence. A 212 generalizability coefficient of .81 was obtained when ratings were averaged over two observers and three trials. A better understanding of the G coefficients derived in the numerous D studies is enhanced by examining the variance components computed in the G studies. Table 27 summarizes these data. The patterns of the contributions of the variance components to total variance were consistent across all analyses. Subjects' scores accounted for the largest portion of variation in all six G studies. This variation is largely attributable to developmental differences across age that are evidenced in the performance disparities between subjects. Given that the purpose of this study was to examine the generalizability of observers in differentiating developmental differences in hopping performance, variance due to subjects is desirable. Random observer error (SOT,e) and error due to a subject/observer interaction constituted the majority of the relative error variance. Some degree of random error is to be expected. When the residual error component is large. however, the relative error term is increased and generalizability is reduced. For each developmental sequence, the random errors for the elementary education observers tended to be slightly higher than the random errors for the kinesiology observers. 213 The subject/observer interaction is a reflection of the degree to which observers score subjects differently and suggests a measure of observer disagreement. All the analyses revealed some degree of difference in the observer ratings, however the variance attributable to a subject/observer interaction was inconsistent between G studies. The elementary education observers had a larger subject/observer error (21.278) in their arm sequence ratings. This component contributed to over half (538) of the relative error variance (40.238). The arm sequence was also the rating system in which their generalizability scores were lowest (see Table 26). Subject/observer error (23.538) was largest in the composite ratings of the kinesiology observers, contributing to almost 2/3 of the relative error variance (37.008). The composite sequence was the rating system in which the generalizability scores for these observers were lowest. Discussion It is clearly evident that the kinesiology observers in this study could more reliably employ the body-part sequences in rating the hopping performances of the children on the data-collection videotape than could the elementary education observers. Their ratings of total body actions. 214 however, were not as generalizable as those of the elementary education observers. These results do not support the hypothesis that states there would be little difference in the generalizability with which kinesiology students and elementary education students rate the hopping performance of children following training. The generalizability of the kinesiology observers' data also differed between the composite sequence and the component sequences. The magnitude of relative error variance in the ratings of the kinesiology observers was considerably higher for the composite sequence (37.008) than for either the arm sequence (28.768) or the leg sequence (20.248). The coefficients of generalizability for the kinesiology observers employing the composite sequence further substantiate this interpretation of the data. The composite G coefficients were less optimal than those for the arm and leg sequence. These results lend partial support to the research hypothesis that the generalizability of total-body scores would be lower and the minimal conditions of measurement more extensive than for either the arm sequence alone or the leg sequence alone. Two plausible explanations can be offered for the differences that exist in the data between the observational groups. The first explanation is in response to several limitations of the study. The observers for the study were 215 paid volunteers from courses taught by the study investigator. Random assignment of observers to observational groups did not take place. A number of rating and training sessions were scheduled for the observational groups in this study. The arm/leg component groups met on seven different occasions. The total-body composite groups met on five different occasions. Because of this extensive meeting schedule, the observational groups were formed according to the availability of the observers. The total number of viewing sessions experienced by each observational group could be offered as an explanation for the differences in the data. Because the number of rating sessions were different, the arm/leg component groups had two more viewing sessions than the total-body composite groups. The additional exposure of the arm/leg component groups to performances of children hopping may have affected the generalizability of their ratings. Moreover, during the rating sessions the observational focus of the raters using the arm/leg component sequences was more directed than the generalized focus of the raters using the total-body composite sequence. The component observational groups had less information on which to focus for any single viewing than did the composite observational groups. It seems logical that it would take observers a greater number of trials to rate motor performance more reliably 216 using a total-body technique than using any single body- segment sequence. While the focus and the number of sessions could account for the differences in the data of the kinesiology observers, in that the ratings of the arm/leg component group were more consistent than the total- body group, it does not account for the differences in the data of the elementary education observers. Although the arm/leg component and the total-body composite groups for the kinesiology students appeared to be synonymous, it is possible that group assignment based upon availability created systematic differences between the two groups. The observers employing the composite sequence expressed some increased confusion at the end of their second training session. This confusion is an antithesis to formal training in viewing and analyzing motor performance. It is possible they were fixating on a single aspect of the hopping performances, rather than observing the movement in its entirety. It also could be that these observers generally had more difficulty, as a group, in seeing and interpreting actions of the body. The composite kinesiology group was the observational group that refused payment for their services. It is doubtful, however, that this gesture is related to their lower generalizability scores. The composite kinesiology observers appeared to be as highly motivated in the study as 217 were any of the other observers. It is important to note that their refusal to receive payment for their services came at the end of the data-collection sessions and was accompanied by their expression of a genuine interest in the study. Interestingly, the study investigator observed greater differences in the composition of the two elementary education observational groups than in the composition of the two kinesiology observational groups. Three of the volunteers in the composite observational group of the elementary education students were between 28 and 34 years of age, whereas all other volunteers in both groups of observers were between 20 and 25 years of age. None of these three volunteers had their teaching certification or revealed teaching or motor skill experiences that differed from the other elementary education observers. Nevertheless, it was considered feasible that their age, or other life experiences, could have favorably affected their interest in and understanding of the study as well as their assessment of the children's hopping performances. The differences between the elementary education observers' data for the three developmental sequences were not that great, however, which implies that the age of the observers did not have an effect on their ratings. On the other hand, it is also possible that the generalizability of the total-body 218 composite ratings for the elementary education observers is inflated. This would account for the fact that they employed the composite sequence as an assessment instrument more reliably than did the kinesiology observers. A second plausible explanation exists for the differences found between the elementary education observers and the kinesiology observers. Most programs of kinesiology offer courses that focus on the anatomy, applied kinesiology, and biomechanics involved in the movements of body segments. Of the ten kinesiology volunteers for this study, 9 had taken a course in anatomy, 4 had taken a biomechanics course, 8 had taken an exercise physiology course, and 5 had taken a course in motor learning/control. Prior to their enrollment in a motor development class that was being held concurrent to this study and that was examining actions of the body from a total movement perspective, it may be that the experiences of the kinesiology students in analyzing human movement emphasized the actions of body segments. Certainly their courses of study would suggest this possibility. If this were the case, the ability of the kinesiology observers to employ the component sequences more reliably seems justified. Perhaps an emphasis on analyzing the movements of body segments affects the ability of kinesiology students to integrate these movements into a total or complete picture 219 of the body in action. This supposition would explain the lower reliability in the composite ratings of the kinesiology observers in this study. The evidence presented by Roberton (1977b, 1982) that identifies differential rates of development for individual body segments suggests that body-part "profiles" are variable. It is important to note that the uncertainty expressed by the composite kinesiology observers occurred at the end of their second session. This was the session in which the hopping performances of children in transition between classical stages of hopping were viewed and discussed. It may be that the child in transition shows less cohesiveness in the movements of the body segments than does a child at discreetly defined levels of development. Individuals who are adept at focusing on the actions of body segments, which appears to be true with the kinesiology students, may have difficulty in rating transitional performances with total-body scoring techniques. Current research in motor development has focused on the integration of the qualitative changes in motor behavior with dynamical systems theory, thereby examining the cohesiveness between body parts within and across levels of development (Roberton & Halverson, 1988). Perhaps the information derived from this line of research will contribute to an understanding of how better to assist 220 observers in focusing on critical features of movement when children are in transition between levels of development. Although the developmental profile research by Roberton (1977b, 1982) has implications for the differences between the observational groups of the kinesiology students, it does not explain the ability of the elementary educators to rate the action of the total body more reliably than the action of the body parts. The data for the elementary education observers does not lend support to the hypothesis that minimal conditions of measurement would be more extensive for the total-body sequence than for either the arm sequence or the leg.sequence. Although one of the elementary education observers was simultaneously enrolled in a course on the theory of coaching, none of the elementary education students had taken courses concerned with analyzing human movement. The kinesiology course in which they were concurrently enrolled emphasized physical education classroom methodology and not the analysis of movement patterns. This is the typical content for the single course in physical education that most elementary education specialists receive. It may be that their lack of training in analyzing movement does not allow them to separate the movement of the entire body into the individual movements of the body segments. Such an hypothesis would support the results of this study and would have 221 implications for the training of elementary education specialists in fundamental motor skill development. The pre-service training implications for practitioners which arise from this line of reasoning are that developmental sequences describing total-body configurations should be emphasized with elementary education students, and body-part sequences emphasized with kinesiology students. The work of Petrakis (1986, 1987) suggests that experienced and inexperienced observers focus on different aspects of human movement. Fishman and Anderson (1971) indicated that researchers must consider the complexity of the observational systems they design with regard to the observers. Radford (1988) emphasized the need to identify those observational strategies that should be taught to both the preservice and the inservice teachers that are ultimately responsible for children's physical education. Future research is needed in order to address fully the practicality of employing developmental sequences as observational systems. The questions of which sequences work best for different groups of observers is an important practical concern, especially when less experienced observers provide the physical education instruction for young children. This is often the situation in early childhood programs of instruction and in elementary physical education classes. 222 The limitations of this research project infer the need for observers to be randomly assigned to observer groups in future studies. The data obtained on the reliability with which hopping can be assessed using the developmental sequencing technique employed in this study should be replicated with random assignment of observers. The work of Ulrich et al. (1988) and of Mosher and Schutz (1983) should be replicated with elementary education students and/or teachers. The generalizability of other proposed developmental sequences (e.g., gallop, catch, kick, strike) should be investigated. Finally, observational studies of this type should be taken into the field where live action movement may be harder to detect than videotaped movement in a controlled setting. Generalizability analysis offers an excellent procedure for examining the use of developmental sequences as observational systems, for exploring the sources of error that arise in the use of these sequences by various categories of practitioners, and for determining the measurement conditions which can be applied in a field setting. CHAPTER V SUMMARY, CONCLUSIONS, PRACTICAL IMPLICATIONS, AND RECOMMENDATIONS The purpose of this study was threefold: (a) to analyze the effects of training observers to rate the hopping performance of children using developmental sequences as assessment criteria; (b) to investigate the generalizability with which upper-division, undergraduate kinesiology students and upper-division, undergraduate elementary education students could use developmental sequences to assess the hopping performance of children: and (c) to examine the minimal conditions of measurement, reflected in the number of observers and the number of trials, required by each observational group to achieve acceptable levels of generalizability when using developmental sequences as assessment instruments. Hopping performance was evaluated by using three developmental sequences that describe changing body configurations during the development of hopping skill. The developmental sequence hypothesized by Haubenstricker, Henn, and Seefeldt (1975) and prelongitudinally validated by Haubenstricker et al. (1989) served as one of the 223 224 measurement instruments in this study. Each step (stage) in this sequence describes changes in the configuration of the total body. The two other developmental sequences employed as assessment instruments in this study describe changes in the configuration of body parts. One of these sequences describes the action of the arms and the other sequence describes the action of the legs. These two sequences were hypothesized and prelongitudinally validated by Halverson and Williams (1985). Generalizability theory (Brennan, 1983: Cronbach et al. 1963, 1972) served as the framework under which the research designs for this project were conceived. GENOVA, a FORTRAN IV ggfleralized Analysis Qf Vsriance for balanced designs (Crick & Brennan, 1983), was selected for the statistical analysis. Sources of measurement error were analyzed by examining the variance components computed through GENOVA. The generalizability of the data and minimal conditions of measurement were determined by examining generalizability coefficients, part of which were derived through GENOVA and part of which were calculated from the GENOVA variance components using the formulas for inter-observer and inter- trial generalizability presented by Godbout and Schutz (1983). The following statements summarize the differences found in measurement variance between the pre-training ratings and the post-training ratings of hopping performance: 1. The percent of total variance attributable to subjects increased for all observers (observers nested within observer category): (a) 31.288 to 61.488 for the arm sequence data, (b) 62.488 to 68.868 for the leg sequence data, (c) 50.018 to 58.748 for the composite sequence data. The percent of total variance due to relative error was reduced for all observers (observers nested within observer category): (a) 29.228 to 18.028 for the arm sequence data, (b) 16.158 to 13.758 for the leg sequence data, (c) 24.048 to 18.858 for the composite sequence data. For the arm sequence, the percent of total variance attributable to subjects increased for each group of observers: (a) 34.888 to 58.378 for the elementary education observers, (b) 36.258 to 70.438 for the kinesiology observers. The percent of total variance due to relative error in the arm sequence ratings was reduced for each group of observers: (a) 62.288 to 40.238 for the elementary education observers, (b) 56.158 to 28.768 for the kinesiology observers. For the leg sequence, the percent of total 226 variance attributable to subjects increased for each observational group: (a) 56.218 to 62.608 for the elementary education observers, (b) 71.578 to 79.118 for the kinesiology observers. 6. The percent of total variance due to relative error in the leg sequence ratings was reduced for each observational group: (a) 35.048 to 29.838 for the elementary education observers, (b) 24.858 to 20.248 for the kinesiology observers. 7. For the composite sequence, the percent of total variance attributable to subjects increased for each group of raters: (a) 49.658 to 63.568 for the elementary education observers, (b) 50.838 to 53.358 for the kinesiology observers. 8. The percent of total variance due to relative error in the composite sequence ratings was reduced for each group of raters: (a) 49.898 to 35.388 for the elementary education observers, (b) 43.478 to 37.008 for the kinesiology observers. The following statements summarize the differences found between the ratings of the elementary education students and the ratings of the kinesiology students: 1. The percent of total variance attributable to subjects in both the arm sequence and the leg 227 sequence data was greater for the kinesiology students than for the elementary education students. Subject variance in the arm sequence ratings for the kinesiology observers was 70.438, whereas it was 58.378 for the elementary education observers. For the leg sequence data, the kinesiology students obtained 79.118 subject variance as opposed to 62.608 for the elementary education students. In both the arm sequence and the leg sequence data, the percent of total variance due to relative error was less for the kinesiology raters than it was for the elementary education raters. The percent of total variance due to relative error in the arm sequence scores was 28.768 for the kinesiology observers and 40.238 for the elementary education observers. For the leg sequence scores, the percent of total variance due to relative error was 20.248 and 29.838, respectively. In contrast to the results of the arm sequence and the leg sequence, the percent of total variance in the ratings of the total-body composite sequence that were attributable to subjects was greater for the elementary education students than for the 228 kinesiology students: (a) 63.568 for the elementary education raters, (b) 53.358 for the kinesiology raters. The percent of total variance due to relative error in the composite sequence ratings was less for the elementary education observers than for the kinesiology observers: (a) 35.388 for the elementary education students, and (b) 37.008 for the kinesiology students. The minimal conditions of measurement that were required to achieve an acceptable level of generalizability (.80) are summarized for each developmental sequence in the following statements: 1. Both elementary education observers and kinesiology observers achieved acceptable generalizability (.80) for measurement conditions in which a single observer rated the leg action of children hopping. For the elementary education group, one observer and five trials were the minimal measurement conditions considered acceptable. For the kinesiology observers, one observer viewing one trial reached an acceptable level of generalizability. Minimal conditions of measurement required to 229 achieve an acceptable level of generalizability when the arm sequence was employed as the measurement criterion were more extensive for the elementary education observers than for the kinesiology observers. For the elementary education observers, a generalizability coefficient of .81 was obtained when ratings were averaged over two observers and three trials, whereas the kinesiology group reached generalizability equal to .87 with these same conditions. A generalizability coefficient equal to .80 was obtained for the kinesiology group under the less limiting measurement conditions of one observer viewing five trials. Acceptable generalizability levels for subjects’ composite scores required extensive measurement conditions for both categories of observers. A generalizability coefficient of .80 was obtained for the kinesiology students when ratings were averaged over two observers and four trials. When ratings were averaged over two observers and two trials, the elementary education observers obtained a generalizability coefficient of .84. It should be noted that the elementary education observers were just short of an acceptable 230 generalizability level (.79) for one observer viewing five trials. Conclusions Three hypotheses were offered for this investigation. Partial support was obtained for two of the hypotheses. One of the hypotheses was not supported. The first hypothesis was concerned with the effects of training observers to use developmental sequences in the assessment of children's hopping performance. For both the kinesiology observers and the elementary education observers, a reduction of measurement error was found in the post-training data for all the sequences. The significance of these reductions was not testable, but post-training trends in the reduction of relative error variance support the hypothesis. The question does remain, however, whether significant changes were made in the observational abilities of the raters or whether the data instead reflect some random variation based on the testing periods. A second post-training assessment session would assist in clarifying this issue. The second hypothesis suggested that following training in the use of the developmental sequences employed in this study, there would be little difference in the 231 generalizability with which elementary education students and kinesiology students could rate the hopping performance of children. This hypothesis was not supported. The kinesiology raters were more consistent than the elementary education raters in employing the arm/leg component part sequences as assessment criteria, whereas the total-body composite sequence was used more consistently by the elementary education observational group than by the kinesiology group. Finally, the third hypothesis for this research project stated that the minimal conditions of measurement required to obtain generalizability of .80 would be more extensive for the total-body sequence than they would be for either the arm sequence or the leg sequence. This hypothesis was based on the assumption that the observational focus of raters using any single, body-part sequence would have less information to process than raters using a total-body approach. Generally, the results from this study supported this hypothesis. For both the kinesiology students and the elementary education students, the leg sequence was found to be employed more reliably with fewer observers and/or trials than either the arm sequence or the composite sequence. The kinesiology students also were able to use the arm sequence more reliability (with fewer observers) than the composite sequence. For the elementary education observers, however, 232 the minimal conditions of measurement for the composite sequence and the arm sequence differed by only one trial. Practical Implications Several concerns exist with regard to the practical implications of this study. These include a) the number of observers generally available in an assessment setting, b) the time frame available for making decisions about a motor performance, and c) the content of coursework in kinesiology, physical education, and elementary education programs. The results of this study indicate that two observers would be necessary in order to reach acceptable levels of generalizability for three of the observational designs in the study. For both the elementary education students and the kinesiology students using the total-body sequence, the minimal conditions of measurement for acceptable generalizability were two observers. The elementary education observers employing the arm sequence also did not obtain acceptable generalizability until two observers were rating the hopping performances. Generally, physical education classrooms employ only one teacher. The results of this study do not appear to support the arm sequence or the total-body sequence as 233 observational assessment systems conducive to an applied setting. The research design for this study did limit the number of trials viewed to five. It is feasible that one observer viewing more than five trials could reliably assess hopping performance. Even these conditions are in conflict, however, with the assessment practices of one observer and three trials that are applied in most physical activity settings. The problems that surround assessment in the physical education classroom are compounded further when one considers that a body-part sequence by itself does not provide a total-body movement profile. Therefore, in order to derive an assessment of total-body action, the minimal conditions of measurement obtained for the arm sequence must be combined with the minimal conditions of measurement obtained for the leg sequence. In accordance with the results from this study, the minimal number of trials necessary for one observer to assess both the arm action and the leg action of a hopping performance is six for the kinesiology observers and would be over ten for the elementary education observers. When the concern is with the efficiency of assessment. the number of trials becomes an important factor in the minimal conditions of measurement. Because this study was limited to five trials, the number of trials necessary for a single observer to assess hopping performance according to the total-body sequence is not available. It is feasible that the total-body composite sequence would be a more time effective assessment instrument than a total-body profile that comprises the results from both arm sequence and leg sequence ratings. However, whether a total-body composite sequence would be more effective in providing instructional feedback is a different question. An important implication from this study is derived from the inference that undergraduate students preparing for a career in kinesiology, physical education, or elementary education need coursework that helps them develop their abilities to observe and analyze movement patterns. Training can be effective in reducing the measurement error inherent in observational assessment. Further research is needed to determine the extent of such training. How much training is necessary for an individual to reach competency- based levels of observational assessment? What percent of training should be devoted to practical application in a child development laboratory or in the field? What percent of training should be devoted to the scientific basis behind observational systems? It is evident that training will enhance the effectiveness with which observers can assess children's motor skill patterns. The importance of this training in 235 kinesiology programs is usually acknowledged. If elementary classroom teachers are responsible for the physical education of children, then their needs for training in the assessment of children's developing movement behaviors must also be addressed. Unfortunately, elementary education students usually receive a single methods class in physical education. Moreover, this class often is devoted to activity ideas and classroom management. Programs of teacher preparation must consider either proposing further coursework in physical education for the elementary education student or methods by which to integrate the analysis of developing motor patterns into the single methods' class that elementary education students receive. Recommendations Although researchers have recommended the use of developmental motor sequences as observational tools by which to facilitate the development of motor skills in children, research to examine the ease with which such sequences can be employed as observational systems by the practitioner has only recently been initiated. The current study was aimed at furthering an understanding of the generalizability with which motor skill sequences can be used to evaluate the motor behavior of children. The 236 limitations of this project infer the need for future research in this area. Based on the results of this investigation, the following recommendations are submitted: 1. The data obtained on the reliability with which hopping can be evaluated using the developmental sequencing techniques employed in this study should be replicated with (a) random assignment of observers to observational groups, (b) a second post-training rating session, (c) ten observational trials per subject, and (d) a lateral and frontal viewing plane. The work of Ulrich et al. (1988) and of Mosher and Schutz (1983) should be replicated with elementary education students and/or teachers. A large number of developmental motor sequences have been proposed for a variety of fundamental motor skills. The generalizability with which practitioners can employ these sequences as assessment criteria when evaluating children's motor behavior should be investigated. Although using videotaped performances to examine the generalizability with which observers evaluate motor behavior is a useful systematic technique, observational studies such as this one should be 237 replicated in field-based settings. The type of training programs and the extent of training that would be required to bring preservice and inservice teachers to competency levels of generalizability in motor skill assessment procedures needs to be determined. Investigations which examine the visual focus of novice and expert observers as they view motor behavior are encouraged. Results from the dynamic systems research, the visual search patterns research, and generalizability analyses of observational systems should be integrated. This approach may provide a foundation for building training programs that facilitate the development of expertise in identifying the critical elements of a developing motor skill. Courses of study in elementary school physical education for preservice or inservice elementary classroom teachers should devote some attention to the identification and evaluation of fundamental motor skill patterns. When time constraints are limited and prior experiences are minimal, total- body configurations probably should be emphasized. In addition to the emphasis that courses in kinesiology place on the analysis of movements 238 from a body-part perspective, studies in kinesiology should provide experiences that allow the student to integrate their knowledge of body parts back into a visual whole. Investigators and hypothesizers of developmental motor sequences should give considerations to wording (or rewording) categorical descriptions using terminology that is easily interpretable by the practitioner. APPENDICES APPENDIX A SUBJECTS' INFORMED CONSENT: INFORMATION LETTER FOR PARENTS AND CONSENT FORM MICHIGAN STATE UNIVERSITY COLLEGE or EDUCATION . SCHOOL or HEALTH [promos EAST MNSIN‘b ° MICHIGAN ' “"2“” cousszuso PSYCHOLOGY AND HUMAN “momma . m SPORTS CIRCLE July 1, 1988 Dear Parents: I am a doctoral student at Michigan State University in physical education and exercise science, with an emphasis in motor development. My dissertation research is examining the reliability with which teachers-in- training can assess the hopping skill of children using two different developmental rating techniques. I am contacting you at this time to request your permission to allow your child to participate as a volunteer in my study. My study is being conducted through the joint cooperation of the Motor Performance Study and the Early Childhood Motor Skills Project at Michigan State University and the Central Branch of the Y.M.C.A. in Lansing, Michigan. The purposes and procedures of the study have been explained to the administrators of each program, and each of them has agreed to participate in the project. This letter provides you with information concerning the purposes and procedures of the study and serves as a request for your permission to allow your child to participate in this study. The results of the study will be made available upon request. There are two purposes for this study. The first is to investigate the accuracy and reliability with which physical education majors and elementary education majors can assess the hopping performance of children. The second is to examine the minimal conditions of measurement (number of observers and number of trials) required for accurate and reliable assessment. The results of the study should be useful in developing standardized instruments and procedures for assessing the fundamental motor skills of children. Data collection for this study will be in two phases. The first phase entails the videotaping of children performing the fundamental motor skill of' hopping. ‘This is the phase of the study for which your child's assistance is requested. The second phase of the study involves training physical education and elementary education majors to assess the hopping performance of children. There will be no direct interaction between your child and the assessors during this second phase of the study. Instead of directly observing your child, the assessors will view two training videotapes and a data-collection videotape that will be dubbed from the videotape made during the first phase of the study. 239 MS U is as A/firwutive Action /Equsl Opportunity Institution Page 2 Painter The children will be videotaped as they hop on their preferred foot no further than 20 feet. Hopping is a fundamental motor skill frequently incorporated into sport and game activities and leisure time activities for children of all ages. There are no unusual risks associated with performing this task. Prior to performing the hopping task, the children will be informed of the purpose of this study, will be given a verbal description and a visual demonstration of the hopping task, and will have a chance to ask questions about any aspect of the study. Following this discussion, each child will be given the opportunity to withdraw from participation in this study. A child may also withdraw from the study at any time during the videotaping session. Each child will be given two trials to practice hopping prior to any videotaping. Five trials will actually be videotaped. Only one child will be videotaped at a time. Children will be given positive feedback about their performance. The identity of each child will be kept strictly confidential and will be ensured by replacing each child's name with an identification number. In order to assure this confidentiality, the identity of the children will not be revealed on the videotape, to the observers in the second phase of the study, or in any reports of research findings. Only group information will be referred to in any publication of the results of this study. It is possible that the videotapes will be used in the future to train physical education and elementary education students and educators in the assessment of hopping, but you can be assured that the identity of your child will remain anonymous. The dates that have been reserved for videotaping are July 12, 13, 18, 19, and 212 All videotaping will take place during the regularly scheduled program hours for both the Michigan State University Motor Programs and the Y.M.C“A. Day Camp. Please read the enclosed parental consent form that outlines the rights of your child with regard to this study. If you agree to your child's participation, please sign the form and have your child return it to his or her instructor as soon as possible. It would be most beneficial if the form were returned no later than Monday, July 11, 1988. If you have any question or concerns about the study, please feel free to contact me at 351-7619 (home) or 355-8323 (work). Your cooperation in this project will be greatly appreciated. The results of studies such as this help to furnish information through which educators may continue to provide a better educational environment for all children. Sincerely, - Student Physical Education and Exercise Science Michigan State University 240 PARENTAL CONSENT FORM Study: A Generalizability Analysis of Observational Abilities in the Assessment of Hopping Using Two Developmental Approaches to Motor Sequencing. Principal Investigator: Mary Painter (child's name - please print) has my permission, as legal parent or guardian, to volunteer for the study being conducted by Mary Esipter on the assessment of children's hopping performance. I have received and understand the following information concerning this study: 1. I have read the information contained in the accompanying letter concerning the proposed project which is being conducted with children attending either the Motor Performance Study or the Early Childhood Motor Skills Study on the Michigan State University campus, or a youth program at the Central Branch of the Y.M.C.A. in Lansing, Michigan. 2. I understand the explanation of the study and I understand what my child's participation will involve. 3. I understand that, at my request, I can receive an additional explanation of the study from the principal investigator. 4. I understand that I am free to ‘withdraW’ my consent and discontinue my child's participation at any time. 5. I understand that my child’s participation is completely voluntary and that he/she is free to discontinue participation at any time. 6. I understand that the results of this study will be treated in strict confidence and that the identity of my son or daughter will remain anonymous. Within this restriction, results of the study will be available at my request. Signature of Parent or Guardian Name of Parent or Guardian - Please Print Date Child's Date of Birth 241 APPENDIX B OBSERVERS' INFORMED CONSENT: LETTER TO OBSERVERS AND CONSENT FORM r—L- . . . . LEE»: L ant‘rSlI)‘ nfC Oloradn at Boulder “11).!!! MW!” HI kmrsiuh I2) ( l"',"H\ “(N :3; szdc' ( .~iuv.:.:n.\()1()\)-IIT‘4l .\ \ .'.r,1.J\J‘.-::.. MEMORANDUM TO: Potential Subjects, Data Collection Phase FROM: Mary A. Painter, Department of Kinesiology TBl-lO6. 492-5209 RE: Research Study -- A Generalizability Analysis of Observational Abilities in the Assessment of Hopping Using Two Developmental Approaches to Motor Sequencing. DATE: February 15, 1989 The purpose of this study is to (a) investigate the accuracy and generalizability with which upper-division kinesiology (physical education) majors and upper-division elementary education majors can use whole-body and body-part developmental motor sequences to assess the hopping performance of children, and (b) to examine the minimal conditions of measurement (number of observers and number of trials) required for both methods to be used reliably by kinesiology majors and by elementary education majors. Students volunteering to serve as subjects for the study will be asked to attend two training sessions and either two or four data-collection sessions. Each training session will last approximately 1 hour and each data-collection session will last no longer than 1! hours. Depending upon the data- collection group to which subjects are assigned, the approximate total time commitment for the study will be either 6 hours over four sessions or 7-8 hours over six sessions. All sessions will be completed within three weeks. This study will aid motor developmentalists in recommending both minimal and optimal measurement conditions necessary for the assessment of children's hopping skill. These recommendations are valuable in the standardization of motor skill assessment instruments and procedures used by individuals concerned with the motor development of children. Your involvement in the study will enhance your understanding of motor development and provide you an opportunity to become involved in a research process. Additionally, you will be paid $3.00 per hour for your time. Please note that there is no risk in your participation. Your anonymity will be ensured by assigning you a subject number that you will use when you complete the data-collection form. Your name and number will be kept strictly confidential. You have the right to withdraw from participation at any point during the study. 242 Page 2 Questions concerning your rights as a subject can be directed to the Human Research Committee at the Graduate School of the University of Colorado and upon request you may receive a COpy of this Institution's General Assurance from the Human Research Committee Secretary, Graduate School, University of Colorado, Boulder, Colorado 80309. Mary Painter will be glad to answer any questions you have about this study. She can be reached at 492-5209 (office) or 499-5981 (residence). 243 SUBJECT CONSENT FORM STUDY: A Generalizability Analysis of Observational Abilities in the Assessment of Hopping Using Two Developmental Approaches to Motor Sequencing. PHASE: Data-collection phase PRINCIPAL INVESTIGATOR: Mary A. Painter " I have read the informational letter concerning the generalizability study to be conducted on the assessment of children hopping. I understand that I am free to withdraw my consent and discontinue my participation at any time. I also understand that the results of the study will be treated in strict confidence and that my identity will remain anonymous. Within these restrictions, I understand that I may request the results of the study and/or an additional explanation of the study upon its completion. I hereby give my consent to participate as a volunteer in the study to be conducted by Mary Painter." Name (please print): Signature: Date: 244 APPENDIX C PAGE 1 OF OBSERVER SCORESHEETS: ARM SEQUENCE LEG SEQUENCE COMPOSITE SEQUENCE OIEEIVEI SCUIEEHEETI PAINTER DISSERTATION menu-ma macro mt STEP! 9151.2 3121 IE4 lilet. inactive t. ee iet m m: high Slight swing. Peep up 8 deem. (be are wings I: to side. flinging block. Held to trout in opposition inactive. of body. to wing leg. (LEFT) 245 m M lotli em suing opposite legs. SUBJECT t DESERVER SCORESNEETS PAINTER DISSERTATION LES ACTION - HOPPINS DATE STEP 1 Sgggort leg - pulled iron floor. Sving leg -- inactive. Noeentery‘ilight 1-2 hops only 2 §gggort lgg - einieel extension. ,Svin I - inactive. Fell t catch repeated hops ,sma Seamus- ore-takeoff extension. Sv n le - eininel swinging or pulping. mu Suggort lgg - Full extension Bell of foot Suing leg -- Full suing through. (LEFT) (LEFT) -e ‘ ”I O H (RIGHT) -e ~ ”0 Q — a. * D ‘ q -o “0 . u "5.300e-15 4 4 “a . ~ E n (LEFT) (RIGHT) SI4£5 OIOERVER SCORESNEETS PAINTER DISSERTATION mm mm - Isms SUBJECT 4 DATE ' 87.34 M1 M3. 5142.1 Inactive Bvigg Igg svin I Bvigg lgg DELLA! Foot in Iront Foot held back Thigh dragged Pendular sving Thigh parallel Thigh angled Reactive sving Ares Stabilized lilat, ares Iilat. ares eggg_ Hands shoulder ht Iilat. passing lilat. sving Ara-leg opposition Elhovs flexed m Ileld Vertical lody - Hig flex. £911 hinieal height slight (vd. lean Forvard lean Forvard lean I PERF. I: (LEFT) Trial 1 ma 2 Ir_il_l 3 ILL“ 4 Trial 3 PERF. 23 (LEFT) Bill 1 Trial 2 LVLOU Int! 4 M 5 PERT. 3: (RIGHT) Mel! Trig) Z M 3 ILL“ 4 Iris 5 £§![g_gi (LEFT) me.) 1 M 2 Trial 3 mm Trial 5 S: (RIGHT) am I Trig) 2 Trial 3 Trig] 4 IL“ 5 SL477 APPENDIX D RAW DATA DISSERTATION DATA ENTRY COMMAND FILE DATA ENTRY CODEBOOK FOR ALL OBSERVER DATA SETS VARIABLE LIST: INPUT FORMAT: VAR LABELS: DAY. SSN. EEl TO EE5, KINEl TO KINES CA,AGE.SEX.ORIG# F1.0.1X.F2.0.1X,6F1.0.6F1.0.6F1.0,6F1.0. 6F1.0,6F1.0,6F1.0,6F1.0,6F1.0,6F1.0,1X, F3.0.1X.F2.1.1X.F1.0.4X.F2.0) DAY PRE-POST ASSESSMENT 1 = PRE-TRAINING RATINGS 2 = POST-TRAINING RATINGS SSN SUBJECT N ' EEl EL ED OBSERVER 1 EE2 EL ED OBSERVER 2 EE3 EL ED OBSERVER 3 EE4 EL ED OBSERVER 4 EE5 EL ED OBSERVER 5 KINEl KINE OBSERVER 1 KINEZ KINE OBSERVER 2 KINEB KINE OBSERVER 3 KINE4 KINE OBSERVER 4 KINES KINE OBSERVER 5 CA AGE IN MONTHS AGE AGE IN YEARS SEX 1 = MALES 2 = FEMALES ORIG# ORGINAL SUBJECT N 248 ”NNDNQNNNNNNNNNN”NNNHHHHHHHHHHHHHHHHHHHH ”HHHHHHHHHHcoceoo°°°~HHHHHHFHHH°°°°°°°°° 90-dou-UNH°\D-~DGU“wNH°O~“0‘“-uupcoug0mfiwufi-fi 2I463 OBSERVER DATA -- ARM SEQUENCE 32333 22334 22322 32223 22222 12223 53332 13323 33333 33333 050 40 1 43333 44453 44454 33444 44544 54434 22225 55555 51414 54445 074 65 2 32334 22333 33443 33233 33223 33333 33343 22323 33223 32322 069 55 2 12333 45545 44455 34444 12222 54445 55455 54444 54445 54445 094 40 2 14222 11222 14121 23222 11111 22222 44444 13343 22242 54444 074 65 2 24333 22232 42321 54454 44442 54343 45554 54455 54454 54555 096 40 2 33233 12133 22321 32222 13213 33323 12112 11112 42232 22222 092 75 1 43233 33122 13112 33223 11112 44233 22542 33432 23441 44455 101 45 1 33333 33334 33333 33333 33333 33333 33333 32322 33334 55555 059 50 2 22323 11332 23333 23333 22333 12322 55333 32322 22323 44543 074 60 1 44223 44445 22244 45444 24244 44344 45555 44355 44245 43454 069 55 1 33332 22334 23322 33333 33342 23333 33333 22322 13322 22222 064 55 1 33333 22333 33433 33333 32332 22222 22333 33333 33433 54545 079 65 2 33333 12323 22322 43333 33333 12222 22222 33323 23222 44543 061 50 1 22222 11121 11221 22332 11222 11211 11122 11444 12224 44544 045 70 1 33333 23333 33333 23333 33333 33333 33333 33333 33333 33333 047 35 2 34444 44344 42423 54554 44444 54444 54454 55444 54455 44445 049 75 1 44433 23222 12221 44433 44334 22232 12222 23323 44444 55454 091 75 2 22222 11112 11114 11111 22222 11111 41111 11122 21222 22333 042 35 2 22242 12121 23242 22223 32444 33233 44444 32332 33442 23232 070 60 1 11111 12211 12111 12222 11111 22222 22222 23322 22222 11111 050 40 1 44444 33333 44444 44445 43444 54544 43333 55434 44444 44444 074 65 2 22222 32332 33222 33322 32222 22332 32222 33323 33333 33333 069 55 2 44444 44444 44444 45454 54444 44444 44343 44444 44444 54444 094 40 2 22422 11112 22222 44434 22424 22222 22222 22222 24444 22222 074 65 2 54544 45544 44452 54455 55555 55554 55554 54554 45554 35554 096 40 2 22213 11111 22212 32222 33222 22222 22222 12222 32222 23332 092 75 1 22222 43333 22212 44432 22222 22222 22222 23232 43422 44444 101 45 1 33332 22222 33333 33333 22233 32333 33333 33333 33333 33333 059 50 2 11222 11112 11112 21322 12233 22222 22222 22322 22222 22222 074 60 1 44444 44444 42242 43332 34443 34444 22222 32322 22222 22222 069 55 1 22333 32332 22332 22323 33332 22222 22222 12222 32332 12222 064 55 1 32333 33333 33233 32222 22222 33333 22232 33333 33333 33333 079 65 2 23222 22333 22222 23323 22223 23322 33333 22322 22222 22322 061 50 1 12222 42444 22222 44422 22222 22222 22222 22222 22222 22222 045 70 1 33333 33333 33333 32333 22222 23333 33333 33233 33333 33333 047 35 2 44443 44455 54444 34443 44333 44444 44444 34443 44444 44444 049 75 1 44444 44444 22222 33433 44444 22322 24224 23323 22434 44444 091 75 2 11221 11112 11111 11111 11212 11111 11111 11111 11111 11111 042 35 2 43444 44444 12111 12233 24442 23232 22222 33333 33332 33332 070 60 1 96 17 19 61 12 45 47 43 20 16 54 59 14 01 30 99 10 60 42 03 96 17 19 61 12 45 47 43 20 16 54 59 14 01 30 99 10 60 42 03 ”nuwnunnweonuwuw””QNNHHHHHHHHHHHHHHHe—IHHHH ”wHHHHHHHHe-Ioooeococonh-HHHHHHHe—Hooooocooo condomgwpe-Iosoeo-.amus-wuwomag¢m-uue—eoougc~m-wnu 9354) OBSERVER DATA -- LEG SEQUENCE 12211 11112 22212 12222 11222 22111 11111 21212 11211 22232 050 40 1 33332 34433 44344 44443 33333 33333 33333 43343 43433 44333 074 65 2 11221 22231 12221 23222 22112 12222 12222 22222 22222 23233 069 55 2 33443 44444 44344 44444 33444 44444 44444 44444 44444 44444 094 40 2 22122 11121 22322 34444 33333 22222 22233 23223 23333 33233 074 65 2 22333 44334 33433 44343 43433 33333 43333 33333 33433 33333 096 40 2 33333 33233 22323 34444 34344 44333 34343 43344 44444 43333 092 75 1 33232 22322 22232 33332 22222 22222 22222 22222 22222 23232 101 45 1 22221 22223 23222 33333 32322 22222 22222 33322 33322 23333 059 50 2 23211 22332 23121 23223 23333 22222 12111 22221 23222 23222 074 60 1 23323 23333 12322 33333 33334 33233 33322 33333 32323 33322 069 55 1 21221 21212 11211 22222 22222 11111 21221 21211 21212 22222 064 55 1 22222 22222 22222 22323 22232 12122 22222 22222 22222 22222 079 65 2 12222 11222 12111 12222 23222 22212 11222 22222 22222 22222 061 50 1 22322 22222 22222 32332 22333 22222 22222 22222 22322 33332 045 70 1 22222 11122 12122 22222 22222 12112 12212 22222 12222 22222 047 35 2 33233 43433 43332 44334 33333 44434 43333 43333 43333 43333 049 75 1 33323 44434 34233 33343 33333 33333 22222 33333 34433 44334 091 75 2 11111 11111 12111 11212 12111 11111 11111 11112 11112 12212 042 35 2 32222 22231 22212 33443 33333 33333 23322 23332 33332 22222 070 60 1 12111 11111 22111 22222 22222 22222 12211 21111 11111 22222 050 40 1 33333 23333 43333 43333 33333 33333 33333 33333 33333 33333 074 65 2 22121 11122 11122 22222 12222 22222 22222 22222 22222 22222 069 55 2 33443 33333 33444 34444 33433 33333 33444 44444 33444 44444 094 40 2 22233 11222 33333 34334 23333 22222 23333 22222 33333 23333 074 65 2 33333 33333 32323 43333 33333 33333 43333 43333 34333 33333 096 40 2 33333 33332 33333 33343 33333 33333 33333 33333 33443 33333 092 75 1 22222 22222 22222 22222 22222 22222 22222 22222 22222 23333 101 45 1 32222 22222 32222 33232 22232 22222 22222 32222 33232 32333 059 50 2 22221 12111 22111 22222 33222 22222 12222 22222 22222 22222 074 60 1 33333 22212 32222 33344 33333 33333 33333 33233 33334 33333 069 55 1 11111 21121 21221 22222 22222 22222 22222 21221 11222 22222 064 55 1 12112 22222 22222 22222 22222 22222 22222 22222 22222 22222 079 65 2 22112 11111 22222 22222 22222 22222 22222 22222 12222 22222 061 50 1 22222 22222 22222 32222 22232 22222 22222 22222 22222 22222 045 70 1 12222 11111 11122 22222 12222 22122 22222 12122 12222 22222 047 35 2 33332 33333 33333 43334 33333 33333 33333 33333 43333 33333 049 75 1 34434 32222 22322 33333 33333 33333 33333 33333 33333 33333 091 75 2 11111 11111 11111 11111 11111 11111 11111 11111 11111 12112 042 35 2 33332 22222 22222 33222 11221 33333 33333 33332 33333 23222 070 60 1 96 17 19 61 12 45 47 43 20 16 54 59 14 01 30 99 10 60 42 03 96 17 19 61 12 45 47 43 20 16 54 59 14 01 30 99 10 60 42 03 ”pumpwunwnunwnpnnweawe-eHHHHHHHHHHHHHHHHHF‘H NHHHHHHHHO—‘HOcooooocowe—IHHe—IHHHe—Ie—IHOOOOQQcog OW“~IQW-WNH°\O~I~IOM‘WNH°\°”do‘m-w”H°Wm~la~m-wN0—¢ 13551 OBSERVER DATA -- COMPOSITE SEQUENCE 22232 22222 22222 11111 32222 11112 22322 33332 22222 33222 050 40 34322 43443 33343 33433 44444 33332 33434 43433 33333 43323 074 65 23333 11211 21221 23222 11122 11222 32332 22222 11122 11112 069 55 44343 44443 44444 43444 44434 33443 44444 33434 43434 33433 094 40 34443 11121 34444 34444 33333 11112 23333 33233 33334 22223 074 65 24334 33323 32333 44444 42344 43333 43433 44443 34433 33432 096 40 23322 43333 33323 32223 32323 22222 33333 22322 21232 34233 092 75 22232 32222 22333 33333 22222 12222 22222 32333 22222 32333 101 45 22322 33312 32222 33333 44422 22122 22233 22333 23222 22322 059 50 24334 23333 32222 22222 22222 12122 22222 23222 22222 22222 074 60 44323 32343 23223 32334 43333 22221 23233 22333 33333 33234 069 55 22123 12111 22222 22222 22122 21222 22121 22222 22222 22122 064 55 22233 12232 22222 22222 22222 22122 22222 22332 12222 12222 079 65 22232 12111 22222 12222 12212 12121 22212 12212 12212 22211 061 50 22232 22232 32223 43223 22222 23222 22222 22221 22332 23333 045 70 11211 11111 11111 11111 11111 11111 21111 11111 11111 11111 047 35 24332 43133 31131 44433 43344 33222 32122 24423 33333 43343 049 75 22223 43423 34333 33333 23222 22232 22222 32323 23322 23323 091 75 12221 22212 11111 11111 11211 11111 12112 12221 21222 22221 042 35 21122 33313 22122 22212 22232 22222 23222 23233 23333 22232 070 60 22222 22222 22222 22222 22222 22333 22222 22222 22222 32222 050 40 44444 33433 44344 43333 44444 44444 33444 33332 33433 44444 074 65 22221 22222 11221 22221 12222 21222 22222 11222 12222 22222 069 55 44444 43444 44444 43333 44444 44444 44444 22222 33433 44444 094 40 33244 33332 33343 23233 23334 22222 33333 22332 12112 33334 074 65 43343 43333 33333 43444 44433 33322 44444 33333 43333 44443 096 40 34324 23333 33344 22222 44444 33333 33333 22222 23333 23333 092 75 34333 33333 33333 33333 33343 33333 33333 33333 22223 33333 101 45 33323 22222 23333 23223 33333 22222 33333 33333 33333 33333 059 50 24222 22222 22222 12112 23222 22222 23333 22222 22222 22222 074 60 24444 33334 43334 34334 44444 34334 44444 22222 33344 22224 069 55 12221 22222 22222 22122 12112 22222 22222 22222 22212 22222 064 55 22332 22222 22233 22222 12223 22222 23333 22222 12222 22222 079 65 32222 11222 22211 22221 12212 22222 12222 22222 22212 22222 061 50 33434 33333 33333 33333 33233 33222 23333 22223 22222 33333 045 70 11111 11111 11111 11111 11111 11111 11111 11111 11111 11111 047 35 22344 32233 33133 43443 44444 43344 44444 22222 43333 44343 049 75 33222 34434 33334 44444 24434 44333 33434 22222 22232 33333 091 75 33323 22222 22222 22222 11111 22222 12222 22222 21212 22222 042 35 0 1 2 2 2 2 2 1 1 2 1 1 1 2 1 1 2 1 2 2 1 1 2 2 2 2 2 1 1 2 1 1 1 2 1 1 2 1 2 2 24242 23322 44322 22222 44422 33343 33333 22222 33333 22222 070 6 1 96 17 19 61 12 45 47 43 20 16 54 59 14 01 30 99 10 60 42 03 96 17 19 61 12 45 47 43 20 16 54 59 14 01 30 99 10 60 42 03 01 02 03 04 05 06 07 04 09 10 11 12 13 14 15 16 17 14 19 20 INTER\INTRA INVESTIGATOR DATA 12211 22222 22222 55444 44444 44444 33333 33333 32332 55555 55555 44444 22222 22424 44444 55554 55555 55555 33233 33232 33232 33322 22222 33243 33333 33333 33333 22222 22222 22222 44444 44444 44444 33333 33333 33333 33333 33333 33333 33332 33333 33333 22222 22222 22222 33333 33333 33333 44444 44444 44444 44444 44444 44444 11111 11111 12211 22222 43434 33332 12532 22222 22222 33222 33333 33333 33333 22222 22222 22222 33333 33333 33343 33333 33333 44434 33333 33333 33333 43333 33333 34333 32222 22222 22222 33333 32222 33333 33333 33333 33333 33333 33334 33334 22222 22222 22222 22222 22222 22222 22222 22222 22222 22222 22222 22232 22222 22222 22222 33333 33333 33333 33333 33333 33333 12222 12212 22222 33333 33333 33333 22222 33322 33332 44444 44444 44444 22222 22222 22222 44444 44444 44444 33333 44444 44444 44444 44444 44444 44444 44444 44444 33333 33333 33333 33333 33333 33333 22222 22222 22222 44444 44444 44444 22222 22222 22222 22222 22222 22222 22222 22222 22222 33333 33333 33333 11111 11111 11111 22222 44444 44444 33333 33333 33333 22222 22222 22222 44444 44444 44444 96 17 17 61 12 45 47 43 20 16 54 59 14 01 30 99 10 60 42 03 DISSERTATION DATA ENTRY COMMAND FILE DATA ENTRY CODEBOOK FOR INTER/INTRA INVESTIGATOR DATA SET VARIABLE LIST: INPUT FORMAT: VARIABLES: SSN, A1, A2, AE, L1, L2, LE, Bl, B2, BE, CA.AGE.SEX.ORIG# F2.0.2X.6F1.0,6F1.0.6F1.0.1X. 6F1.0,6F1.0,6F1.0.1X,6F1.0.6F1.0.6F1.0.2X. F3.0,1X.F2.1.1X.F1.0.7X.F2.0) SSN SUBJECT N A1 1$T RATING ARMS A2 2ND RATING ARMS AE EXPERT RATING ARMS L1 lsT RATING LEGS L2 2ND RATING LEGS L3 EXPERT RATING LEGS Bl 1$T RATING BODY B2 2ND RATING BODY BE EXPERT RATING BODY CA AGE IN MONTHS AGE AGE IN YEARS SEX 1 = MALES 2 = FEMALES ORIGN ORGINAL SUBJECT # 253 APPENDIX E GISTUDY ANALYSIS OF VARIANCE TABLES 254 Table 28: Analysis of Variance Table for Pre-Training Ratings (All Observers Using Arm Sequence) F-STAT 2F FACET DF SS MS F-STATISTIC MUM DEM Subjects (8) 19 430.73100 22.670058 10.06681 OF 19 166 Category (C) 1 38.02500 38.02500 Obs:Cat (O:C) 8 63.67600 7.95950 4.22809 OF 8 129 Trials (T) 4 3.89600 0.97400 1.56387 9F 4 44 SC 19 61.67500 3.24605 1.53304 OF 19 158 SO:C 152 292.96400 1.92739 5.61729 152 608 ST 76 50.74400 0.66768 1.94593 76 608 CT 4 2.68000 0.67000 1.37214 QF 4 36 OT:C 32 9.54400 0.29825 0.86923 32 608 SCT 76 40.52000 0.53316 1.55386 76 608 Residual 608 208.61600 0.34312 (SOT:C,e) TOTAL 999 1203.07100 NOTE: For generalizability analyses, F-statistics should be ignored. Table 29: Analysis of Variance Table for Post-Training Ratings (All Observers Using Arm Sequence) g-SIAT DF FACET DF SS MS F-STATISTIC MUM DEM Subjects (S) 19 700.38400 36.86232 26.79460 9! 19 167 Category (C) 1 0.10000 0.10000 Obs:Cat (O:C) 8 18.72400 2.34050 1.91621 9! 8 138 Trials (T) 4 2.72400 0.68100 2.20934 9F 4 43 SC 19 37.98000 1.99895 1.60422 OF 19 148 SO:C 152 187.03600 1.23050 7.15079 152 608 ST 76 24.11600 0.31732 1.84401 76 608 CT 4 0.66000 0.16500 0.92410 QF 4 24 OT:C 32 5.21600 0.16300 0.94724 32 608 SCT 76 14.26000 0.18763 1.09038 76 608 Residual 608 104.62400 0.17208 (SOT:C.e) TOTAL 999 1095.82400 NOTE: For generalizability analyses, F-statistics should be ignored. 255 Table 30: Analysis of Variance Table for Pre-Training Ratings (All Observers Using Leg Sequence) F-STAT DE FACET DF 85 MS F-STATISTIC NUM DEN Subjects (S) 19 497.59600 26.18926 38.15811 OF 19 146 Category (C) 1 0.01600 0.01600 Obs:Cat (O:C) 8 41.58000 5.19750 9.81148 OF 8 101 Trials (T) 4 1.76600 0.44150 1.20162 OF 4 51 SC 19 9.18400 0.48337 1.02519 OF 19 116 SO:C 152 76.50000 0.50329 3.18684 152 608 ST 76 25.91400 0.34097 2.15905 76 608 CT 4 1.21400 0.30350 1.98913 OF 4 18 OT:C 32 5.90000 0.18437 1.16747 32 608 SCT 76 9.58600 0.12613 0.79867 76 608 Residual 608 96.02000 0.15793 (SOT:C.e) TOTAL 999 765.27600 NOTE: For generalizability analyses, F-statistics should be ignored. Table 31: Analysis of Variance Table for Post-Training Ratings (All Observers Using Leg Sequence) z-sgAT DF FACET DF SS MS F-STATISTIC NUM DEN Subjects (S) 19 407.55500 21.45026 41.66696 OF 19 169 Category (C) 1 6.24100 6.24100 Obs:Cat (O:C) 8 23.56400 2.94550 6.96119 OF 8 127 Trials (T) 4 0.27000 0.06750 0.28140 OF 4 59 SC 19 7.65900 0.40311 1.07566 OF 19 132 SO:C 152 59.15600 0.38918 4.84647 152 608 ST 76 15.65000 0.20592 2.56431 76 608 CT 4 0.19400 0.04850 0.48590 OF 4 21 OT:C 32 3.65600 0.11425 1.42274 32 608 SCT 76 5.00600 0.06587 0.82025 76 608 Residual 608 48.82400 0.08030 (SOT:C,e) TOTAL 999 577.77500 NOTE: For generalizability analyses, F-statistics should be ignored. 256 Table 32: Analysis of Variance Table for Pre-Training Ratings (All Observers Using Total-Body Sequence) F-STAT 2F FACET OF SS HS F-STATIETIC MUM DEN Subjects (S) 19 399.87500 21.04605 21.10158 OF 19 146 Category (C) 1 4.76100 4.76100 Obs:Cat (O:C) 8 21.10400 2.63800 2.77538 OF 8 118 Trials (T) 4 0.22000 0.05500 0.16424 OF 4 34 SC 19 15.09900 0.79468 0.83080 OF 19 140 SO:C 152 139.53600 0.91800 4.11659 152 608 ST 76 22.98000 0.30237 1.35591 76 608 CT 4 0.76400 0.19100 0.64960 OF 4 29 OT: 32 8.17600 0.25550 1.14574 32 608 SCT 76 19.87600 0.26153 1.17276 76 608 Residual 608 135.58400 0.22300 (SOT:C.e) TOTAL 999 767.67500 NOTE: For generalizability analyses, F-statistics should be ignored. Table 33: Analysis of Variance Table for Post-Training Ratings (All Observers Using Total-Body Sequence) [-STAT OF FACET DF SS MS F-STATISTIC MUM DEN Subjects (8) 19 471.47500 24.81447 25.83071 OF 19 167 Category (C) 1 3.48100 3.48100 Obs:Cat (O:C) 8 37.02400 4.62800 5.48051 OF 8 134 Trials (T) 4 1.57000 0.39250 1.33170 OF 4 47 SC 19 17.17900 0.90416 1.09759 OF 19 142 SO:C 152 125.61600 0.82642 5.80052 152 608 ST 76 21.03000 0.27671 1.94219 76 608 CT 4 0.61400 0.15350 0.97249 OF 4 23 OT:C 32 5.13600 0.16050 1.12652 32 608 SCT 76 10.62600 0.13982 0.98134 76 608 Residual 608 86.62400 0.14247 (SOT:C,e) TOTAL 999 780.37500 NOTE: For generalizability analyses, F-statistics should be ignored. 257 Table 34: Analysis of Variance Table for Pre-Training Ratings (Elementary Education Observers Using Ara Sequence) F-STAI DF FACET DP SS MS F-STATISTIC NUN DEN Subjects (S) 19 201.64800 10.61305 5.58516 OF 19 91 Observers (O) 4 12.42800 3.10700 1.82015 OF 4 65 Trials (T) 4 4.74800 1.18700 1.82571 OF 4 30 SO 76 124.13200 1.63332 5.27615 76 304 ST 76 43.81200 0.57647 1.86220 76 304 OT 16 6.13200 0.38325 1.23802 16 304 Residual 304 94.10800 0.30957 (SOT.e) TOTAL 499 487.00800 NOTE: For generalizability analyses, F-statistics should be ignored. Table 35: Analysis of Variance Table for Post-Training Ratings (Elementary Education Observers Using Ara Sequence) [-STAT Dz FACET DF SS MS F-STATISTIC NUM DEN Subjects (8) 19 383.22200 20.16958 12.05925 OF 19 83 Observers (O) 4 13.29200 3.32300 2.13394 OF 4 69 Trials (T) 4 1.33200 0.33300 1.05856 OF 4 23 SO 76 119.10800 1.56721 7.14805 76 304 ST 76 24.66800 0.32458 1.48041 76 304 OT 16 3.34800 0.20925 0.95439 16 304 Residual 304 66.65200 0.21925 (SOT,e) 'TOTAL 499 611.62200 NOTE: For generalizability analyses. F-statistics should be ignored. 258 Analysis of Variance Table for Pre-Training Ratings (Kinesiology Observers Using Arm Sequence) Table 36: z-SIAT DF FACET DF SS MS F-STATISTIC MUM DEN Subjects (S) 19 290.75800 15.30305 6.19765 OF 19 86 Observers (O) 4 51.24800 12.81200 6.22530 OF 4 62 Trials (T) 4 1.82800 0.45700 0.99144 OF 4 25 SO 76 168.83200 2.22147 5.89765 76 304 ST 76 47.45200 0.62437 1.65760 76 304 OT 16 3.41200 0.21325 0.56614 16 304 Residual 304 114.50800 0.37667 (SOT,e) TOTAL 499 678.03800 NOTE: For generalizability analyses, F-statistics should be ignored. Table 37: Analysis of Variance Table for Post-Training Ratings (Kinesiology Observers Using Arm Sequence) F-ST;1 DF FACET DF SS MS F-STATISTIC MUM DEN Subjects (S) 19 355.14200 18.69168 19.69100 OF 19 82 Observers (O) 4 5.43200 1.35800 1.53337 OF 4 69 Trials (T) 4 2.05200 0.51300 2.97891 OF 4 22 SO 76 67.92800 0.89379 7.15559 76 304 ST 76 13.70800 0.18037 1.44401 76 304 OT 16 1.86800 0.11675 0.93469 16 304 Residual 304 37.97200 0.12491 (SOT.e) TOTAL 499 484.10200 NOTE: For generalizability analyses, F-statistics should be ignored. 259 Table 38: Analysis of Variance Table for Pre-Training Ratings (Elementary Education Observers Using Leg Sequence) [-SZAT OF FACET DF SS MS F-STATISTIC NUM DEN Subjects (S) 19 235.95800 12.41881 18.97836 OF 19 71 Observers (O) 4 30.46800 7.61700 12.92400 OF 4 49 Trials (T) 4 1.96800 0.49200 2.51088 OF 4 15 SO 76 47.53200 0.62543 3.08009 76 304 ST 76 17.63200 0.23200 1.14256 76 304 OT 16 2.67200 0.16700 0.82245 16 304 Residual 304 61.72800 0.20305 (SOT.e) TOTAL 499 397.95800 NOTE: For generalizability analyses, F-statistics should be ignored. Table 39: Analysis of Variance Table for Post-Training Ratings (Elementary Education Observers Using Leg Sequence) F-STAT DF FACET DF SS MS F-STATISTIC NUM DEN Subjects (S) 19 208.79200 10.98905 19.05993 OF 19 90 Observers (O) 4 21.53200 5.38300 10.17479 OF 4 63 Trials (T) 4 0.27200 0.06800 0.30279 OF 4 29 SO 76 37.42800 0.49247 4.73892 76 304 ST 76 14.28800 0.18800 1.80907 76 304 OT 16 2.24800 0.14050 1.35199 16 304 Residual 304 31.59200 0.10392 (SOT.e) TOTAL 499 316.15200 NOTE: For generalizability analyses, F-statistics should be ignored. 260 Table 40: Analysis of Variance Table for Pre-Training Ratings (Kinesiology Observers Using Leg Sequence) F-STAT Dz FACET DF SS MS F-STATISTIC NUM DEN Subjects (S) 19 270.82200 14.25379 28.31163 OF 19 95 Observers (O) 4 11.11200 2.77800 5.90931 OF 4 49 Trials (T) 4 1.01200 0.25300 0.78074 OF 4 32 SO 76 28.96800 0.38116 3.37898 76 304 ST 76 17.86800 0.23511 2.08422 76 304 OT 16 3.22800 0.20175 1.78852 16 304 Residual 304 34.29200 0.11280 (SOT.e) TOTAL 499 367.30200 NOTE: For generalizability analyses, F-statistics should be ignored. Table 41: Analysis of Variance Table for Post-Training Ratings (Kinesiology Observers Using Leg Sequence) F-STA FACET DF SS MS F-STATISTIC NUM DEN Subjects (8) 19 206.42200 10.86432 34.71027 OF 19 83 Observers (O) 4 2.03200 0.50800 1.60146 OF 4 64 Trials (T) 4 0.19200 0.04800 0.41701 OF 4 23 SO 76 21.72800 0.28589 5.04364 76 304 ST 76 6.36800 0.08379 1.47818 76 304 OT 16 1.40800 0.08800 1.55246 16 304 Residual 304 17.23200 0.05668 (SOT.e) TOTAL 499 255.38200 NOTE: For generalizability analyses, F-statistics should be ignored. 261 Table 42: Analysis of Variance Table for Pre-Training Ratings (Elementary Education Observers Using Total-Body Sequence) L'fiT—AT—Dl FACET 0? SS MS F-STATISTIC NUM DEN Subjects (8) l9 237.94200 12.52326 9.86104 OF 19 78 Observers (O) 4 3.19200 0.79800 0.61604 OF 4 64 Trials (T) 4 0.17200 0.04300 0.11090 OF 4 18 SO 76 92.16800 1.21274 4.89266 76 304 ST 76 23.18800 0.30511 1.23092 76 304 OT 16 5.28800 0.33050 1.33337 16 304 Residual 304 75.35200 0.24787 (SOT,e) TOTAL 499 437.30200 NOTE: For generalizability analyses, F-statistics should be ignored. Table 43: Analysis of Variance Table for Post-Training Ratings (Elementary Education Observers Using Total-Body Sequence) F-STAT DF FACET DF SS MS F-STATISTIC NUM DEN Subjects (S) 19 290.18200 15.27274 18.26394 OF 19 85 Observers (O) 4 6.15200 1.62800 2.23223 OF 4 56 Trials (T) 4 1.37200 0.34300 1.09677 OF 4 25 SO 76 54.92800 0.72274 3.75114 76 304 ST 76 23.26800 0.30616 1.58902 76 304 OT 16 3.18800 0.19925 1.03415 16 304 Residual 304 58.57200 0.19267 (SOT.e) TOTAL 499 438.02200 NOTE: For generalizability analyses, F-statistics should be ignored. 262 Table 44: Analysis of Variance Table for Pre-Training Ratings (Kinesiology Observers Using Total-Body Sequence) F-STAT DF FACET DF SS MS F-STATISTIC MUM DEN Subjects (S) 19 177.03200 9.31747 13.62361 OF 19 76 Observers (O) 4 17.91200 4.47800 7.39393 OF 4 50 Trials (T) 4 0.81200 0.20300 0.84177 OF 4 19 SO 76 47.36800 0.62326 3.14570 76 304 ST 76 19.66800 0.25879 1.30615 76 304 OT 16 2.88800 0.18050 0.91101 16 304 Residual 304 60.23200 0.19813 (SOT.e) TOTAL 499 325.91200 NOTE: For generalizability analyses, F-statistics should be ignored. Table 45: Analysis of Variance Table for Post-Training Ratings (Kinesiology Observers Using Total—Body Sequence) {-STAT DF FACET DF SS MS F-STATISTIC NUM DEN Subjects (S) 19 198.47200 10.44589 11.01658 OF 19 78 Observers (O) 4 30.51200 7.62800 7.94932 OF 4 75 Trials (T) 4 0.81200 0.20300 1.45164 OF 4 18 SO 76 70.68800 0.93011 10.07957 76 304 ST 76 8.38800 0.11037 1.19606 76 304 OT 16 1.94800 0.12175 1.31941 16 304 Residual 304 28.05200 0.09228 (SOT.e) TOTAL 499 338.87200 NOTE: For generalizability analyses, F-statistics should be ignored. BIBLIOGRAPHY BIBLIOGRAPHY Allison, P. C. (1987). What and how preservice physical education teachers observe during an early field experience. Research Quarterly, §§(3), 242-249. Arend, S., & Higgins, J. R. (1976). A strategy for the classification, subjective analysis, and observation of human movement. Journal of Human Movement Studies, s, 36-52. Armstrong, C. W. (1977). Skill analysis and kinesthetic experience. In R. E. Stadulis (Ed.), Research and practice in physical education. (Pp. 13-18). Champaign, IL: Human Kinetics. Armstrong, C. W., & Hoffman, S. J. (1979). Effects of teaching experience, knowledge of performer competence, and knowledge of performance outcome on performance error identification. Research Quarterly. 99(3). 318-327. Atkinson, R. K. (1924). A motor efficiency study of 8,000 New York city high school boys. American Physical Education Review, 99(1), 56-59. Atkinson, R. K. (1925). A study of the athletic ability of high school girls. American Physical Education Review, 99(7). 389-399. Barrett, K. R. (1977). We see so much but perceive so little: Why? In L. I. Gedvilas and M. E. Kneer (Eds.). Proceedings of the NCPEAMZNAPECW national conference, (pp. 180-190). Chicago: University of Illinois at Chicago Circle. Barrett, K. R. (1979a). Observation for teaching and coaching. Journal of Health, Physical Education, and Recreation, 99(1), 23-25. Barrett. K. R. (1979b). Observation of movement for teachers--A synthesis and implications. Motor Skills: Theory into Practice, 9(2), 67-76. 263 In. 264 Barrett, K. R., Allison, P. C., 8 Bell, R. (1987). What preservice physical education teachers see in an unguided field experience: A follow-up study. Journal of Teaching in Physical Education, 1, 12-21. Baumgartner, T. A. (1969). Estimating reliability when all test trials are administered on the same day. Research Quarterly, 99, 222-225. Bayless, M. A. (1981). Effect of exposure to prototypic skill and experience in identification of performance error. Perceptual and Motor Skills, 99, 667-670. Bayley, N. (1935). The development of motor abilities during the first three years. Monographs pf theysgciety for Research_9n Child Development, 9, 1-26. Beck, M. C. (1965). The pa ath of the center of gravity during running_9n boys, grades one to six. Unpublished doctoral dissertation, University of Wisconsin, Madison. Bell, R. Barrett, K. R., 8 Allison, P. C. (1985). What preservice physical education teachers see in an unguided, early field experience. Journal of Teaching in Physical Education, 9(2), 81-90. Berk, R. A. (1979). Generalizability of behavioral observations: A clarification of interobserver agreement and interobserver reliability. American Journal of Mental Deficiengy, 83(5), 460- -472. Biscan, D. V., & Hoffman, S. J. (1976). Movement analysis as a generic ability of physical education teachers and students. Research Quarterly, 91(2), 161-163. Bliss, J. G. (1927). A study of progression based on age, sex, and individual differences in strength and skill. American Physical Education Review, 99, 11-21, 85-99. Booth, C. L., Mitchell, S. K., & Solin, F. K. (1979). The generalizability study as a method of assessing intra- and interobserver reliability in observational research. Behavior Research Methods and Instrumentation, 99(5), 491-494. Brace, D. K. (1927). Measuring Motor Ability. New York: A. S. Barnes. 265 Brainerd, C. J. (1978). The stage question in cognitive-developmental theory. The Behav9ors9 and Brain Sciences, 9, 173-213. Branta, C., Haubenstricker, J., & Seefeldt, V. (1984). Age changes in motor skills during childhood and adolescence. In R. L. Terjung (Ed.). Eggrcise and sport science review: Volsme 99. (pp. 467-520). Lexington, MA: D. C. Heath. Branta, C., Kiger, J., & Yager, M. (1985, April). Profiles of theymptorjperformsnce of preschool age children. Paper presented at the national convention of the American Alliance of Health, Physical Education, Recreation, and Dance, Atlanta, GA. Brennan, R. L. (1983). Elements of generalizabilisy theory. Iowa City, Iowa: ACT Publications. Brennan, R. L., & Kane, M. R. (1979). Generalizability theory: A review. In R. E. Traub, Newsgirectionssfor testing andsmeasprement: Methodolgical developments, No. 4. (pp. 33-53). San Francisco: Jossey-Bass. Broer, M. R. (1960). Efficiency of humss movement (lst ed.). Philadelphia: W. B. Saunders. Broer, M. R. (1966). Efficiency of human movement (2nd ed.). Philadelphia: W. B. Saunders. Broer, M. R. (1973). Efficiency of human movement (3rd ed.). Philadelphia: W. B. Saunders. Broer, M. R., & Zernicke, R. F. (1974). Efficisscy o; Human Movement (4th ed.). Philadelphia: W. B. Saunders. Burnside. L. (1927). Coordination in the locomotion of infants. Genetic Psycholpgy Monographs, 9, 283-372. Cardinet, J., Tourneur, Y., & Allal, L. (1976). The symmetry of generalizability theory: Applications to educational measurement. Journal of Educational Measurement, 99(2), 119-135. Cardinet, J., Tourneur, Y., a Allal, L. (1981). Extension of generalizability theory and its applications in education measurement. Journal of Educational Measuremenp, 99(4), 183-204. 266 Carpenter, A. (1942). The measurement of general motor capacity and general motor ability in the first three grades. Researchsguarterly, 99(4), 444-465. Cheffers, J. (1977). Observing teaching systematically. Quest, _9, 17-27.' Clark, J. E. & Phillips, S. J. (1985). A developmental sequence of the standing long jump. In J. E. Clark & J. H. Humphrey (Eds.), Motor develgppenty Currentsselected research: Volume 1 (pp. 73-85). Princeton: Princeton Book. Clark, J. E., Stamm, C. L., & Urquia, M. F. (1979). Developmental variability: The issue of reliability. In G. G. Roberts & K. M. Newell (Eds.), Psychology of motor behavior and sport--1978. Champaign, IL: Human Kinetics. Clarke, H. H. & Petersen, K. H. (1961). Contrast of maturational, structural, and strength characteristics of athletes and nonathletes 10-15 years of age. Research Quarterly, 99(2), 163-176. Clarke, H. H. (1971). Physical and motor tests in the Medford bgys' growth study. Englewood Cliffs, N.J.: Prentice-Hall. Clouse, F. C. (1959). A kinetic analysis of the dsvelopment of the_;snninggpattern of preschool boys. Unpublished doctoral dissertation, University of Wisconsin, Madison. Cowan, E. A., & Pratt, B. M. (1934). The hurdle jump as a developmental and diagnostic test of motor coordination for children from three to twelve years of age. Child Qevelopment, 9, 107-121. Cratty, B. J. (1975). Remedial motor ssfivity for childsen. Philadelphia: Lea & Febiger. Cratty, B. J. (1979). Perceptual snd motor:gevelopment in infants and chilgren. Englewood Cliffs, N.J.: Prentice-Hall. Crick, J. E., & Brennan, R. L. (1983). Manual for GENOVA: A GENeralized ana9ysis Of VAriance system, Version 2.1. Iowa City, Iowa: American College Testing Program. 267 Cronbach, L. J., Gleser, G. C., Nanda, H., 8 Rajaratnam, N. (1972). The dependability ofybehavioral measurements: Theory of generalizability for spores and profiles. New York: Wiley. Cronbach, L. J., Rajaratnam, N., a Gleser, G. C. (1963). Theory of generalizability: A liberalization of reliability theory. The British Journal of Statistical Psychology, 99, 137-163. Cunningham, B. V. (1927). An experiment in measuring gross motor development of infants and young children. Journal of Educational Psychology, 99, 458-464. Curtis, E. M. (1975). Norms of an eleven year study of selected motorgperforpsnce tests for Hawaiian chi99reny 5-12 years of age. Unpublished manuscript, Curriculum and Instruction Department, University of Hawaii at Manoa. Deach, D. F. (1951). Genetic development of motor skills of children two through six years of age. Dissertstion Abstracts Internstional, 99(2). 287-288. (University Microfilms No. 2390). Denckla, M. B. (1974). Development of motor co-ordination in normal children. Developmental Medicine and Child Neurology, 99, 729-741. DeOreo, K. & Keogh, J. (1980). Fundamental motor skills: The performance of children. In C. B. Corbin, 9 textbook ofssstor development (pp. 76-97). Dubuque, Iowa: Wm. C. Brown. Dittmer, J. (1962). A kinematic analysis of the development of the running pattern of grade school girls and certain factors whish distinguish good from poor performances at the observed ages. Unpublished masters thesis, University of Wisconsin. Madison. Dohrmann, P. (1964). Throwing and kicking ability of 8-year-old boys and girls. Research Quarterly, 99(4). 464-471. Dusenberry, L. (1952). A study of the effects of training in ball throwing by children ages three to seven. Research Quarter9y, 99 (1), 9-14. 268 Earls, N. F. (March, 1975). A three year study of changes in physical fitness, motor skills and physical education knowlegge o; elementary children. Paper presented at the annual convention of the American Alliance of Health, Physical Education, and Recreation, Atlantic City, N.J. Early Childhood Motor Skills Development Study, 1978-1984. (1985). Standards of performance on selected fundamental motor skills_§or preschool age children. Unpublished tabulated data, Michigan State University, East Lansing. Ebel, R. L. (1951). Estimation of the reliability of ratings. Psychometriss, 99(4), 407-424. Eckert, H. M. (1987). Motor development (3rd ed.). Indianapolis: Benchmark Press. Ellis, J. D., Carron, A. V., & Bailey, D. A. (1975). Physical performance in boys from 10 through 16 years. Human Biolggy, 91(3), 263-281. Erikson, E. H. (1982). Life cycle. In J. K. Gardner (Ed.). Readings in develspmental psychology, 2nd ed. (pp. 3-12). Boston: Little, Brown, & Co. (Reprinted from D. L. Sills, Ed., Internssional Encyclopedia of Social Sciencssy Vo9pme 9 pp. 286-292. Copyright 1968 by Crowell Collier & Macmillan). Espenschade, A. (1940). Motor performance in adolescence. Monographs of theysociety 99; Research in Child Development, 9(1, Serial No. 24), 1-126. Espenschade, A. (1947). Development of motor coordination in boys and girls. Research Quarterly, 99(1), 30-44. Espenschade, A. (1960). Motor development. In W. R. Johnson (Ed.), Science and medicine of exerc9se and sports (pp. 419-439). New York: Harper and Brothers. Espenschade, A. S., E Eckert, H. M. (1980). Motor development (2nd ed.). Columbus: Charles E. Merrill. Fishman, S. E., 8 Anderson. W. G. (1971). Developing a system for describing teaching. Quest, 99, 9-16. Flavell, J. H. (1972). An analysis of cognitive-developmental sequences. Genetic pSYchology Monographs, 99, 279-350. 269 Flavell, J. H. (1985). Cognitive sevelopment (2nd ed.). Englewood Cliffs. N.J.: Prentice-Hall. Fountain. C., Ulrich, B., Haubenstricker, J., Seefeldt, V. (1981, March). Relationship of developmental stage and running velocity 9n children99 1L9:54years of age. Paper presented at the Midwest convention of the American Alliance of Health, Physical Education, Recreation and Dance, Rosemont, IL. Fortney, V. L. (1983). The kinematics and kinetics of the running pattern of two-, four-, and six-year-old children. Research Quarterly, 99(2). 126-135. Frankenburg, W. K. and Dodds, J. B. (1967). The Denver Developmental Screening Test. Journal of Pediatrics, 19(2). 181-191. Freud, S. (1961). Three contripptions to the theory of sex. New York: E P. Dutton. (Original work published in 1905) Frick, R., 8 Semmel, M. I. (1978). Observer agreement and reliabilities of classroom observational measures. Review of Educational Research, 99(1), 157-184. Gallahue, D. L. (1982). Understanding motor development in children. New York: John Wiley 8 Sons. Gesell. A. (1929). Maturation and infant behavior pattern. Psychologicsl Review, 99, 307-319. Gesell, A. (1939). Reciprocal interweaving in neuromotor development. The Journal of Comparative Neurology, 19(2). 161-180. Gesell, A. (1940). The §9rst five years of life. New York: Harper 8 Brothers. Gesell, A. (1946). The ontogenesis of infant behavior. In L. Carmichael (Ed.), Manual of Child Psycholggy (pp. 295-331). New York: John Wiley and Sons. Girardin, Y., 8 Hanson, D. (1967). Relationship between ability to perform tumbling skills and ability to diagnose performance errors. Resesrch Quarterly, 99(4). 556-561. 270 Glassow, R. B., 8 Kruse, P. (1960). Motor performance of girls age six to fourteen years. Research Quarterly. 99(3). 426-433. Gleser, G. C., Cronbach, L. J., 8 Rajaratnam, N. (1965). Generalizability of scores influenced by multiple sources of variance. Psychometriks, 99(4), 395-418. Godbout, P., 8 Schutz, R. W. (1983). Generalizability of ratings of motor performances with reference to various observational designs. Research Quarterly, 99(1). 20-27. Godfrey, B. B.. 8 Kephart, N. C. (1969). Movemsnt patterns and motor education. New York: Appleton-Century- Crofts. Gutteridge, M. V. (1939). A study of motor achievements of young children. Archives of Psychology, 99(244), 1-178. Halverson, H. M. (1932). A further study of grasping. Journal of General Psychology, 1, 34-64. Halverson, L. E. (1958). A comparison of performance of kindergarten children in the take-off phase of the standing broad jump. Dissertation spstrsgts 9sternations9, 99, 2052-2051. (L. C. Card No. Mic 58-1903). Halverson, L. E. (1971). A real look at the young child. Journal of Health, Physical Education, and Recreation, 99(5). 31-33. Halverson, L. E., Roberton, M. A., 8 Harper, C. J. (1973). Current Research in Motor Development. Journal of Research and Development in Education, 9(3), 56-70. Halverson, L. 8 Williams, K. (1985). Developmental sequences for hopping over distance: A prelongitudinal screening. Research Quarterly, 99(1). 37-44. Hanson, M. R. (1965). Motor performance testing of elementary school age children. Dissertation Abstracts International, 91, 01A. 271 Haubenstricker, J., Branta, C., 8 Seefeldt, V. (1983, May). Preliminary validation of developmental seguences for throwing and catching. Paper presented at the annual convention of the North American Society for the Psychology of Sport and Physical Activity, East Lansing, MI. Haubenstricker, J., Branta, C., Seefeldt, V., Brakora, L., 8 Kiger, J. (1989. April). Prelongitudinal screening of a developmental seguence for hopping. Paper presented at the national convention of the American Alliance of Health, Physical Education, Recreation, and Dance, Boston, MA. Haubenstricker, J., Branta, C., Ulrich, D., Brakora, L., 8 E- Lotfalian, A. (1984. February). ngntitative and gualitative analysis of jumping behavior in young children. Paper presented at the midwest convention of American Alliance of Health, Physical Education, Recreation and Dance, Indianapolis, IN. Haubenstricker, J., 8 Ewing, M. (1985, April). Predictipg motor perforpgnce from changes in body size and shape. Paper presented at the national convention of the American Alliance of Health, Physical Education, Recreation and Dance, Atlanta, GA. Haubenstricker, J., Henn, J., 8 Seefeldt, V. (1975). A developmental seguence for hopping. Unpublished manuscript, Michigan State University, East Lansing. Haubenstricker, J., 8 Seefeldt, V. (1974, March). Seguential progression in fundamental motor skills of children with learning disabilities. Paper presented at the international conference of the Association for Children with Learning Disabilities, Houston, TX. Haubenstricker, J., 8 Seefeldt. V. (1976). A developmental seguence for skipping. Unpublished manuscript, Michigan State University, East Lansing. Haubenstricker, J. 8 Seefeldt, V. (1986).‘ Acquisition of motor skills during childhood. In V. Seefeldt (Ed.). Physical activity and well-being (pp. 41-102). Reston, Virginia: AAHPERD. 272 Haubenstricker, J., Seefeldt, V., 8 Branta, C. (1983, April). Preliminary validation of a developmental seguence for the standing long jump. Paper presented at the national convention of the American Alliance of Health, Physical Education, Recreation and Dance, Minneapolis, MN. Haubenstricker, J., Seefeldt, V., Fountain, C., 8 Sapp, M. (1981, March). A developmental seguence for kicking. Paper presented at the Midwest convention of the American Alliance of Health, Physical Education, Recreation, and Dance, Chicago, IL. Haywood, K. M. (1986). Life span motor development. Champaign, IL: Human Kinetics. Hellebrandt, F. A., Rarick, G. L., Glassow, R., 8 Carns, M. L. (1961). Physiological analysis of basic motor skills. American Jomrnal of Physical Medicine. 52. 14-25. Herkowitz, J. (1978). Developmental task analysis: The design of movement experiences and evaluation of motor development status. In M. Ridenour (Ed.). Motor development: Issues and applications (pp.139-164). Princeton, N.J.: Princeton Book Co. Hoffman, S. J. (1974). Toward taking the fun out of skill analysis. Journal of Health, Physical Education and Recreetion, £§(9), 74-76. Hoffman, S. J. (1977a). Observing and reporting on learner responses: The teacher as a reliable feedback agent. In L. I. Gedvilas 8 M. E. Kneer (Eds.), Proceedings of the NAPECW/NCPEAM National Conference (pp. 153-160). Chicago: University of Illinois at Chicago Circle. Hoffman, S. J. (1977b). Toward a pedagogical kinesiology. Quest , mg, 38-48. Hoffman, S. J. (1983). Clinical diagnosis as a pedagogical skill. In T. J. Templin 8 J. K. Olson (Eds.), Teaching in Physical Education (pp. 35-45). Champaign, IL: Human Kinetics. Holstein, C. B. (1976). Irreversible, stepwise sequence in the development of moral judgment: A longitudinal study of males and females. Child Development, g1, 51-61. 273 Jenkins, L. M. (1930). A comparative study of motor achievements of children of five, six, and seven years of age. Contributions to Education No. 414. New York: Teachers College, Columbia University. Johnson, G. B. (1932). Physical skill tests for sectioning classes into homogeneous units. Research Quarterly. 1(1). 128-136. Johnson. R. D. (1962). Measurements of achievement in fundamental skills of elementary school children. Research Quarterly. 11(1). 94-103. Jones, M. C. (1926). The development of early behavior patterns in young children. Journal of Genetic Psychology, 11, 537-585. Kane, R. J. and Meredith, H. V. (1952). Ability in the standing broad jump of elementary school children 7, 9, and 11 years of age. Research Quarterly. 22(1). 198-208. Keogh, J. (1965). Motor performance of elementary school children. (USPHS Grants MH 08319-01 and HD 01059). Department of Physical Education, University of California, Los Angeles. Keogh, J. (1970). Motor performance test data for ‘ elementary school children. Research Quarterly, 11(4). 600-602. Keogh, J.. 8 Sugden, D. (1985). Movement skillggevelopment. New York: MacMillan. Kiger, J. (1985). Film tracings of developmental stages for selected fundamental motor skills. Unpublished graphics, Michigan State University, East Lansing. Kohlberg. L. (1963). The development of children's orientations toward a moral order-~Part I. Sequence in the development of moral thought. Vita Humana, g, 11-33. Kroll. W. (1962). A note on the coefficient of intraclass correlation as an estimate of reliability. Research Quarterly, 11(2), 313-317. Kurtines, R., 8 Greif, E. B. (1974). The development of moral thought: Review and evaluation of Kohlberg's approach. Psychological Bulletin, 11(8). 453-470. 274 Langendorfer, S. (1987a). A prelongitudinal test of motor stage theory. Research Quarterly. 11(1). 21-29. Langendorfer, S. (1987b). Prelongitudinal screening of overarm striking development performed under two environmental conditions. In J. E. Clark 8 J. H. Humphrey (Eds.), Advances in_motoragevelopment research: Volame 1 (pp. 17-47). New York: AMS. Latchaw, M. (1954). Measuring selected motor skills in fourth, fifth, and sixth grades. Research Quarterly, 11(4). 439-449. Locke, L. F. (1972). Implications for physical education. Research Quarterly, 11(3), 374-386. Malina, R. M., 8 Rarick, G. L. (1973). In G. L. Rarick (Ed.), Physical Activity: Human Growth and Development. New York: Academic Press. McCaskill, C. L., 8 Wellman, B. L. (1938). A study of common motor achievements at the preschool ages. Child Development, 1(2), 141-150. McClenaghan, B. A., 8 Gallahue, D. L. (1978). Fundamental movement: A gavelopmentalaand remedial approach. Philadelphia, W. B. Saunders. McCloy, C. H. (1934). The measurement of general motor capacity and general motor ability. Research Quarterly, §(2) I 46-62. McGraw, M. B. (1932). From reflex to muscular control in the assumption of an erect posture and ambulation in the human infant. Child Developmeng. 1. 291-297. McGraw, M. B. (1939a). Swimming behavior of the human infant. Journal of Peaiatrica, 11, 485-490. McGraw, M. C. (1939b). Later development of children specially trained during infancy: Johnny and Jimmy at school age. Child Development, 19, 1-19. McGraw. M. B. (1941a). Development of neuro-muscular mechanisms as reflected in the drawling and creeping behavior of the human infant. Journal of Genetic Psychology, aa, 33-111. 275 McGraw, M. B. (1941b). Neural maturation as exemplified in the reaching-prehensile behavior of the human infant. Journal of Psychology. 11. 127-141. McGraw, M. B. (1943). The neuromuscular maturation of the human infant. New York: Columbia University Press. McGraw, M. B. (1946). Maturation of behavior. In L. Carmichael (Ed.), Manual of Child Psychology (pp. 332-369). New York: John Wiley and Sons. Medley, n. M.. & Mitzel, n. a. (1958). Application of analysis of variance to the estimation of the reliability of observations of teachers' classroom behavior. Journal of Experimental Education, 11, 23-35. Michigan Education Assessment Program (1984). Physical Education Assessment Administration Manual, 1984-1985. Lansing, MI: Michigan State Board of Higher Education. Mielke, D., 8 Chapman, D. (1987). Effectiveness of education majors in assessing children on the test of gross motor development. Perceptual and Motor:§killa, £1. 1249-1250. Miller, S., Haubenstricker, J., 8 Seefeldt, V. (1977). Standards of performance in selected motor skills. Unpublished tabulated data, Michigan State University, East Lansing. Mitchell, S. K. (1979). Interobserver agreement, reliability, and generalizability of data collected in observational studies. Psychological Bulletin, 11(2). 376-390. Morrison, C., 8 Reeve, J. (1986). Effect of instructional units on the analysis of related and unrelated skills. Perceptual and Motor Skills, 11, 563-566. Morrison, C. S., Reeve, E. J., 8 Harrison. J. (1984, February). The effect of two methods of teaching skill analysis and skillgperformance. Paper presented to the Southern District, American Alliance of Health, Physical Education, Recreation and Dance, Biloxi, MS. Morris, A.M., Williams, J.M., Atwater, A. E., 8 Wilmore, J. H. (1982). Age and sex differences in motor performance of three through six year old children. Research Quarterly, 11(3), 214—221. 276 Mosher, R.E., 8 Schutz, R. W. (1983). The development of a test of overarm throwing: An application of generalizability theory. Canadian Jourmal of App1iea Sport Science, 1(1), 1-8. Osborne, M. M., 8 Gordon, M. E. (1972). An investigation of the accuracy of ratings of a gross motor skill. Research Quarterly. 11(1). 55-61. Payne, V. G., 8 Isaacs, L. D. (1987). Human motor gevelopment: A lifespan Approach. Mountain View, CA: Mayfield. Petrakis, E. (1986). Visual observation patterns of tennis teachers. Research Quarterl , 11(3), 254-259. Petrakis, E. (1987). Analysis of visual search patterns of dance teachers. Journal of Teaohimg in Physical Education, 1, 149-156. Piaget, J. (1960). The definition of stages of development. In J. M. Tanner 8 B. Inhelder (Eds.), Discussions on child developmento Volume IV (Pp. 116-135). New York: International University Press. Piaget, J. (1983). Piaget's theory. In P. H. Mussen (Ed.). Hamdbook of Child Paychology. 4th eo,: Volume 1 (pp. 103-128). New York: John Wiley 8 Sons. (Reprinted from Carmichael's Manual of Child Psychology. 1970). Pinard, A., 8 Laurendeau. M. (1969). ”Stage" in Piaget's cognitive-developmental theory: Exegesis of a concept. In D. Elkind 8 J. Flavell (Eds.), Studies in cognitive gavelopment (pp. 121-170). New York: Oxford University Press. Poe, A. (1970). Development of vertical jumping skill in children. Unpublished manuscript, University of Wisconsin, Madison. Radford, K. W. (1988). Observation -- a neglected teaching skill. CAHPER Journal, November/December, 45-47. Rarick, G. L. (1973). Stability and change in motor abilities. In G. L. Rarick (Ed.). Physical activity: Human growth and development (pp. 201-226). New York: Academic Press. 277 Rarick, G. L., 8 Smoll. F. L. (1967). Stability of growth in strength and motor performance from childhood to adolescence. Human Biology, 11, 295-306. Rentz, R. R. (1980). Rules of thumb for estimating reliability coefficients using generalizability theory. Educational and Psychological Measurement, 11, 575-592. Roberton, M. A. (1977a). Motor stages: Heuristic model for research and teaching. Paper presented to the National Convention of the National Association for Physical Education for College Men and Women, Orlando, FL. Roberton, M. A. (1977b). Stability of stage categorizations across trials: Implications for the ‘stage theory' of overarm throw development. Journal of Human Movement Studies, 1, 49-59. Roberton, M. A. (1978a). Longitudinal evidence for developmental stages in the forceful overarm throw. Journal of Roman Movement Studies, 1, 167-173. Roberton, M. A. (1978b). Stability of stage categorizations in motor development. In D. A. Landers 8 R. W. Christina (Eds.), Psychology of motor behavior and aport,o1917 (pp. 494-506). Champaign, IL: Human Kinetics. Roberton, M. A. (1978c). Stages in motor development. In M. V. Ridenour (Ed.). Motor development: Issues and applicaogons (pp. 63-81). Princeton, NJ: Princeton Book. Roberton, M. A. (1982). Describing ‘stages' within and across motor tasks. In J. A. S. Kelso 8 J. E. Clark (Eds.), gheyaeye1opment ofymovement cont§o1 and co-ordination (pp. 293-307). New York: John Wiley 8 Sons. Roberton, M. A. (1987). Developmental level as a function of the immediate environment. In J. E. Clark 8 J. H. Humphrey (Eds.), Advances in motor development research: Volume 1 (pp. 1-15). New York: AMS Press. Roberton, M. A. (1989). Future directions in motor development research: Applied aspects. In J. Skinner (Ed.), Future directions in_aport and exercise research. Champaign, IL: Human Kinetics. 278 Roberton, M. A., 8 Halverson, L. E. (1984). Developing children-their changing movement: A guide for teachers. Philadelphia: Lea 8 Febiger. Roberton, M. A., 8 Halverson, L. E. (1988). The development of locomotor coordination: Longitudinal change and invariance. Journal of Motor_1ehavior, 11(3), 197-241. Roberton, M. A. 8 Langendorfer, S. (1980). Testing motor development sequences across 9-14 years. In C. Nadeau. W. Halliwell. 8 K. Newell (Eds.), Psychology of motor behavior and sportI 1979 (pp. 269-279). Champaign, IL: Human Kinetics. Roberton, M. A., Williams, K., 8 Langendorfer, S. (1980). Pre-longitudinal screening of motor development sequences. Research Quarterly. 11(4). 724-731. Robinson. S. M. (1974). Visual assessment of children's gross motor patterns by adults with backgrounds in teacher education. Unpublished doctoral dissertation, University of Wisconsin. Madison. Safrit, M. J., Atwater, A. E., Baumgartner, R. A., 8 West, C. (1976). goliability theory. Washington, D. C.: American Alliance for Health, Physical Education, and Recreation. Sapp. M. N. (1980). Developmental seguence of galloping. Unpublished manuscript, Michigan State University, East Lansing. Seefeldt, V. (1972a, May). A researcher's view: Motor development. Paper presented at the Gladys B. Bassett Lecture Series, The University of Wisconsin. Madison. Seefeldt, V. (1972b, March). Developmental seguence of catching skill. Paper presented at the National Convention of the American Alliance of Health, Physical Education, and Recreation National Convention, Houston, TX. Seefeldt, V. (1980).- Developmental motor patterns: Implications for elementary school physical education. In C. Nadeau, W. Halliwell. K. Newell. 8 G. Roberts (Eds.), Psychology of motor behavior and sportI 1979 (pp. 314-323). Champaign, IL: Human Kinetics. 279 Seefeldt, V., 8 Haubenstricker, J. (1974). A developmental seguence for kicking. Unpublished manuscript, Michigan State University, East Lansing. Seefeldt, V., 8 Haubenstricker, J. (1976a). A developmental seguence for punting. Unpublished manuscript, Michigan State University, East Lansing. Seefeldt, V., 8 Haubenstricker, J. (1976b). A developmental seguence foryatriking. Unpublished manuscript. Michigan State University, East Lansing. Seefeldt, V., 8 Haubenstricker, J. (1976c). A developmenta1 seguence for throwing. Unpublished manuscript, Michigan State University, East Lansing. Seefeldt, V. 8 Haubenstricker, J. (1982). Patterns, phases, or stages: An analytical model for the study of developmental movement. In J. A. S. Kelso 8 J. E. Clark (Eds.), Theoaevelopment of movement control and co-ordination (pp. 309-318). New York: John Wiley 8 Sons. Seefeldt, V., Reuschlein, S., 8 Vogel, P. (1972, March). geguencimg motor akil1a,within the physical educatiom curriculum. Paper presented at the national convention of the American Alliance of Health, Physical Education and Recreation, Houston, TX. Seils, L. G. (1951). The relationship between measures of physical growth and gross motor performance of primary-grade school children. Research Quarterly, 11(2). 244-260. Shavelson, R. J., 8 Webb, N. M. (1981). Generalizability theory: 1973-1980. 1£itiah Journal of Mathematical and Statistical Psychology, 11, 133-166. Shavelson, R. J., Webb, N. M., 8 Rowley. G. L. (1989). Generalizability theory. American Psychologist, 11(6), 922-932. Shirley, M. M. (1931). The first two years, a study of twenty-five babies: Volume 1. Postural and locomotor govelopment. Minneapolis: University of Minnesota. Sinclair, C. (1974). Movement and movement patterns of early childhood. Research in‘Eoucation Richmond, VA: Virginia State Department of Education. (Microfiche No. ED 084-014). 280 Smoll, F. L. (1979, April). Developmental kinesiology: Toward a subdiscipline focusing on motogyaevelopment. Paper presented at the CIC Symposium on Motor Skill Development, Iowa City, IOWA. Stamm, C. L., 8 Moore. J. E. (1980). Application of generalizability theory in estimating the reliability of a motor performance test. Research Quarterly, (2), 382-388. Strasser, B. (1967). A conceptual model of instruction. Journal of Teacher Education, 11(1), 63-74. Taylor, J. L. (1979). Development of the physical education observation instrument using generalizability study theory. Research Quarterly, 11(3), 468-481. Touwen, B. C. L. (1979). Examination of the child with minor neurologioa1 dysfunction,o1nd ed. (pp. 79-80). Philadelphia: J. B. Lippincott. Turiel, E. (1969). Developmental processes in the child's moral thinking. In P. Mussen, J. Langer, 8 M. Covington (Eds.), Trends and issues in developmental psychology. New York: Holt, Rinehart, 8 Winston. Ulrich, B. D. (1987). Developmental perspectives of motor skill performance in children. In D. Gould 8 M. R. Weiss (Eds.), Advances in pediatric sport sciences: Volume 2, Behayioral issues (pp. 167-186). Champaign, IL: Human Kinetics. Ulrich, D. A., Riggen, K. J., Ozmun, J. C., Screws, D. P., 8 Cleland, F. E. (In press). Assessing movement control in children with mental retardation: A generalizability analysis of novice and competent observers. Research Quarterly. Ulrich, D. A., Ulrich, B. D., & Branta, c. F. (1988). Developmental gross motor skill ratings: A generalizability analysis. ResearchZQuarterly. §2(3). 203-209. Vilchkovsky, E. S. (1972). Motor development in pre-school and school age children. gmeory and Practice of Phyaica} Culture, 1(46), 29-33. Vincent, M. F. (1968). Motor performance of girls from twelve through eighteen years of age. Research Quarterly, 11(4), 1094-1100. 281 Way, D., Haubenstricker, J., 8 Seefeldt, V. (1979). Performance standards of aelected motor skills for primary grade children. Unpublished tabulated data, Michigan State University, East Lansing. Wellman, B. L. (1937). Motor achievements of preschool children. ChiLQhood Edaoation, 11(7). 311-316. Wickstrom. R. L. (1983). Fundamental motor patterns (3rd ed.). Philadelphia: Lea 8 Febiger. Wickstrom, R. L. (1987). Observations on motor pattern development in skipping. In J. E. Clark 8 J. H. Humphrey (Eds.), Advances in motor development research: Volume 1 (pp. 50-60). New York: AMS. Wild, M. R. (1938). The behavior pattern of throwing and some observations concerning its course of development in children. Research Qaarter1y, 1(3), 20-24. Williams, H. G. (1983). Perceptual and motor gevelopment. Englewood Cliffs, N.J.: Prentice-Hall. Williams, K. (1980). Developmental characteristics of a forward roll. Research Quarterly, 11(4), 703-713. Wohlwill, J. F. (1973). The study of behavioral development. New York: Academic Press.