. .- AJstn u .urcr. r . fl 4.4%... u... {Who 5 . :n n . 3.3....— . .L...A . .... 5‘29, “am a» amarnw r. v; v: is. r... . .24 ,P... .9 “3.5”. .7 . .2 “La, .aa .. .,, I. WE... i? an... 4.4»! my: 4 <..,..._..,fim.uw.~,§?y? 9mm; _ .,,,....,,hwd,nmflh4,~.aflm.m%. 6.“? . ‘ .VWrWrawflnwflmv . , . .. . Jun]. “maév .tuzwau... Kwumn..? :33. 1‘!- ’ v.13 nu .. I s... g. "2:5. ' 1 - "4%. '1’! .1 4.6.. “ 924.... Ara . .2 r. . c k». f 5: 1 .. , 5.?uarrafx.:.;.¥ any“. , .2222 7"lf. 3.9.!) O!!! ‘ u .3 Ear: :23... (a: r». .13 .. , ..> 332343 «fl... {firtilfdl 11.1.5033? L... :3. .4.7;....,..... 3 $31?! 3111...: ; . 123$. .. >1) 1.1.»! I 2...? $3... 33.». . (1...: Air... ufiiummmfiwauq ..,....\ 3 .x a 32453.:«9‘1vyt «7‘-.. (.9 r!» ‘ iv. ‘11 it. It This is to certify that the thesis entitled A MONTE CARLO INVESTIGATION OF THE ANALYSIS OF VARIANCE APPLIED TO NON-INDEPENDENT BERNOULLI VARIATES presented by John Draper has been accepted towards fulfillment of the requirements for Ph.D. Education degree in_____ /L,LQ thfiL Andrew C. Porter Major professor Date May 20, 1971 0-169 ABSTRACT A MONTE CARLO INVESTIGATION OF THE ANALYSIS OF VARIANCE APPLIED TO NON—INDEPENDENT BERNOULLI VARIATES By John Draper The anaLysis of data from a repeated measures type of experimental design was considered for the case in which each repeated measure score was obtained as the sum of a set of evaluations for responses to a set of items. The analysis was considered separately for the case in which the items were included as a factor in the design and for the situation in which the items were ignored as a factor in the design. It was shown that whether items were considered as a factor in the experimental design or not, they could provide a non—null source of variation, which if present would be confounded with the source of variation for repeated measures effects. The suggestion was made that the inclusion of items as a factor in the design and employment of the ”quasi—F" statistic might result in an appropriate test for repeated measures effects, if the response evaluations could be considered independent and normally distrib— uted. However, it was noted that a series of evaluations on a John Draper single subject are seldom independent and are frequently zeros and ones corresponding to incorrect and correct responses, that is, data which could be modeled by a vector of non-independent Bernoulli variates rather than a vector of independent normal variates. Because others had had suc- cess in the application of the Analysis of Variance (ANOVA) to independent Bernoulli variates, it was considered inter— esting and important to attempt to determine if ANOVA could be appropriately applied to the analysis of non-independent Bernoulli variates, particularly with respect to the sub- jects by repeated measures design with items either nested within or crossed with repeated measures. A mathematical modeling of the situation of interest was undertaken, an algorithm was devised, and a computer program was written to simulate zero-one type data with any given desired consistent covariance structure and parameter configuration. For a given parameter configuration a Monte Carlo procedure was employed to determine the appropriate— ness of ANOVA for the analysis. Then the obtained empirical distributions of variance ratio tests were compared to Theoretical F—distributions, with respect to the probability of a type I error or relative power, for null effect or non- null effect conditions respectively. Particular attention was paid to the empirical distributions of the regular variance ratio test for repeated measures, the "quasi-F" test for repeated measures, and the regular variance ratio test for the subjects by repeated measures interaction. John Draper There were 720 cases or parameter configurations which were investigated. For all cases investigated the number of items associated with a repeated measure and the number of repeated measures were fixed at three and four respectively. The items provided either a null source of variation or a non—null source of variation and were either crossed with or nested within the repeated measures. The number of subjects varied from 4 to 12. The probability of a one in the zero- one data was either .5, .2, or .1. The degree of subject heterogeneity was one of four values. And the effects of repeated measures and the subject by repeated measure inter- action were either, both null or separately non—null. The results indicate that the "quasi-F" should not be applied to the type of data investigated, that the power of variance ratio test on non—independent zero-one data is approximately half that for normal data, that in the absence~ of a confounded non—null source of variation the regular variance ratio test for repeated measures is appropriate given the number of subjects is large enough, and that the variance ratio test for the subjects by repeated measures interaction is appropriate only when the probability of a one in the data is close to .5. A MONTE CARLO INVESTIGATION OF THE ANALYSIS OF VARIANCE APPLIED TO NON-INDEPENDENT BERNOULLI VARIATES By John Draper A THESIS Submitted to Michigan State University in partial fulfillment of the requirements for the degree of DOCTOR OF PHILOSOPHY College of Education 1971 ACKNOWLEDGMENTS I would like to express my sincere gratitude to my adviser, Dr. Andrew C. Porter, who served as chairman of my dissertation committee. His comments and aid in the revi- sion of earlier drafts of this work were invaluable. I wish to thank the members of my committee Drs. Joe L. Byers, Maryellen McSweeney, William H. Schmidt, and Robert G. Staudte, Jr., for their assistance. I further wish to thank my colleagues in the Office of Research Consultation, particularly David Wright and Howard Teitlebaum, for their help and patient listening. Finally I would like to thank my wife, Margaret, for her help, support, and understanding throughout this entire project. ii TABLE OF CONTENTS Chapter I. INTRODUCTION . . . . . . . . . . . . . . . . . II. MODEL DEVELOPMENT . . . . . . . . . . . . . . III. A CONSIDERATION OF THE ROBUSTNESS OF THE F-TEST WITH RESPECT TO DEPENDENT BERNOULLI VARIABLES . . . . . . . . . . . . . . . . . IV. METHOD OF DATA GENERATION AND CASES GENERATED V. RESULTS OF THE MONTE CARLO SIMULATIONS . . . . VI. IMPLICATIONS AND CONCLUSIONS . . . . . . . . . BIBLIOGRAPHY . . . . . . . . . . . . . . . . . . . . APPENDICES A. Frequency data for variance ratio tests other than those included in Chapter V . . . , . . B. Empirical values of correlations between mean squares in ratios other than those presented in Chapter V O O O O O O 0 O O O O O O C O C. The listing of the computer program employed in the Monte Carlo procedure . . . . . . . . iii 37 145 54 110 116 118 126 132 LIST OF TABLES Page Sources of Variation, Degrees of Freedom and Expected Mean Squares for Data from a Design Such as Hovland's . . . . . . . . . . . . . . . 4 Sources of Variation, Degrees of Freedom, and Expected Mean Squares for the Suggested Design . 5 Sources of Variation, Degrees okoreedom, and Expected Mean Squares for the Suggested Design when Paired Associates are Fixed . . . . . . . . 9 Sources of Variation, Degrees of Freedom, and Expected Mean Squares for Design 1 . . . . . . . 14 Sources of Variation, Degrees of Freedom, and Expected Mean Squares for Design 2 . . . . . . . 14 Sources of Variation, Degrees of Freedom, and Expected Mean Squares for Design 3 . . . . . . . 15 The Empirical Frequencies, in a-lOOO Rejection Region, of the F for Repeated Measures, Based on Normal and Dichotomous Data, Items Crossed, Interaction and Repeated Measure Effects Both Null . . . . . . . . . . . . . . . . . . . . . . 58 The Empirical Frequencies, in a-lOOO Rejection Regions, of the F for Repeated Measures, Based on the Dichotomous Data Items Nested, Inter- action and Repeated Measure Effects Both Null . 63 The Empirical Frequencies, in aolOOO Rejection Regions, of the F for Repeated Measures, Based on Normal and Dichotomous Data, Items Crossed, Repeated Measure Effects Non-null, Interaction Null . . . . . . . . . . . . . . . . . . . . . . 67 iv 10. ll. 12. l3. 14. 15. l6. 17. 18. The Empirical Frequencies, in a'lOOO Rejection Regions, of the F for Repeated Measures, Based the Dichotomous Data, Items Nested, Repeated Measure Effects Non—null, Interaction Null ... The Empirical Frequencies, in a-lOOO Rejection Regions, of the Quasi—F for Repeated Measures, Based on Normal and Dichotomous Data, Items Crossed, Interaction and Repeated Measures Effects Both Null . . . . . . . . . . . . . . The Empirical Frequencies, in a°1000 Rejection Regions, of the Quasi-F for Repeated Measures, Based on Dichotomous Data, Items Nested, Interaction and Repeated Measure Effects Both Null . . . . . . . . . . . . . . . . . . . . . on The Empirical Frequencies, in the a'lOOO Rejection Region, of the Quasi-F for Repeated Measures, Based on Normal and Dichotomous Data, Items Crossed, Repeated Measure Effects Non-null, Interaction Null . . . . . . . . . . . . . . The Empirical Frequencies, in a°1000 Rejection Regions, of the Quasi-F for Repeated Measures, Based on Dichotomous Data, Items Nested, Repeated Measure Effects Non-null, Inter- action Null . . . . . . . . . . . . . . . . . The Empirical Frequencies, in a°1000 Rejection Regions, of the F and Quasi-F for Repeated Measures, Based on Normal and Dichotomous Data, Items Crossed, Twelve Subjects . . . . . The Empirical Frequencies, in a°lOOO Rejection Regions, of the F and Quasi—F for Repeated Measures, Based on the Normal and Dichotomous Data, Items Nested, Twelve Subjects . . . . . The Empirical Frequencies, in a'lOOO Rejection Regions, of the F for the Subjects by Repeated Measures Interaction, Based on the Normal and Dichotomous Data, Items Crossed, Interaction and Repeated Measures Effects Both Null . . The Empirical Frequencies, in a°1000 Rejection Regions, of the F for the Subjects by Repeated Measures Interaction, Based on the Normal and Dichotomous Data, Items Nested, Interaction and Repeated Measure Effects Both Null . . 68 74 75 81 82 84 85 87 88 19. 20. 21. 22. 23. 24. 25. 26. 27. The Empirical Frequencies, in a-lOOO Rejection Regions, of the F for the Subjects by Repeated Measures Interaction, Based on the Normal and Dichotomous Data, Items Crossed, Interaction Effects Non-null, Repeated Measures Null . . . The Empirical Frequencies, in a°1000 Rejection Regions, of the F for the Subjects by Repeated Measures Interaction, Based on the Normal and Dichotomous Data, Items Nested, Interaction Effects Non-null, Repeated Measures Null . . . The Correlations Between Mean Squares for Repeated Measures and the Mean Square Error Associated, Based on Dichotomous Data, Inter— action and Repeated Measure Effects Both Null The Correlations Between the Mean Squares for Subjects by Repeated Measures Interaction and the Mean Square Error Associated, Based on the Dichotomous Data, Interaction and Repeated Measures Effects Both Null ... . . . . . . . . The Empirical Frequencies, in a-lOOO Rejection Regions, of the F for Subjects Based on Normal and Dichotomous Data, Items Crossed, Inter- action and Repeated Measures Effects Both Null . . . . . . . . . . . . . . . . . . The Empirical Frequencies, in c.1000 Rejection Regions, of the F for Subjects, Based on Normal and Dichotomous Data, Items Nested, Interaction and Repeated Measure Effects Both Null . . . . . The Empirical Frequencies, in a-lOOO Rejection Regions, of the F for Subjects, Based on Normal and Dichotomous Data, Items Crossed, Repeated Measure Effects Non-null, Interaction Effects Nu11 O O O O O O O O O O O O O O O O O O O O Q The Empirical Frequencies, in a-lOOO Rejection Regions, of the F for the Items by Repeated Measures Interaction, Based on the Normal and Dichotomous Data, Items,Nested, Interaction and Repeated Measure Effects Both Null . . . . . . The Empirical Frequencies, in c.1000 Rejection Regions, of the F for Items, Based on the Normal and Dichotomous Data, Items Crossed, Interaction and Repeated Measure Effects Both Null . . . . . . . . . . . . . . . . . . . . . vi 93 94 104 105 118 119 120 121 122 28. 29. 30. 31. 32. 33. 34. 35. 36. The Empirical Frequencies, in 0-1000 Rejection Regions, of the F for Items, Based on the Dichotomous Data, Items Crossed, Repeated Measure Effects Non-null, Interaction Null . The Empirical Frequencies, in a°1000 Rejection Regions, of the F for Items, Based on the Dichotomous Data, Items Nested, Interaction and Repeated Measure Effects Both Null . . . The Empirical Frequencies, in a°1000 Rejection Regions, of the F for Items, Based on the Normal and Dichotomous Data, Items CroSsed, Interaction and Repeated Measure Effects Both Null, Twelve Subjects . . . . . . . . . . . The Empirical Frequencies, in a-lOOO Rejection Regions, of the F for Items, Based on Normal and Dichotomous Data, Items Nested, Interaction and Repeated Measure Effects Both Null, Twelve Subjects . . . . . . . . . . . . . . The Correlations Between the Mean Square Items by Repeated Measures Interaction and the Mean Square Subjects by Items by Repeated Measures Interaction Based on the Dichotomous Data, Interaction and Repeated Measure Effects Both Null . . . . . . . . . . . . . . . . . The Correlations Between the Mean Squares for Subjects and the Mean Square Error Associated, Based on the Dichotomous Data, Interaction and Repeated Measure Effects Both Null . . . The Correlations Between the Mean Squares for Subjects and the Mean Square Error Associated, Based on the Dichotomous Data, Interaction and Repeated Measure Effects Both Null . . . The Correlations Between the Mean Squares for Items and the Mean Squares for Subjects by Items Interaction Based on the Dichotomous Data, Items Crossed, Repeated Measure Effects are Non-null and Interaction Effects are Null . . . . . . . . . . . . . . . . . . The Correlations Between the Mean Squares for Subjects and the Mean Squares for Subjects by Items Interaction Based on the Dichotomous Data, Items Crossed, Repeated Measure Effects are Non-null and Interaction Effects are Null . . . . . . . . . . . . . . . . . . vii 123 124 125 125 128 129 130 131 131 The interaction the number of data in Table The interaction LIST OF FIGURES of the probability of a one and subjects, with respect to the 7 . . . . . . . . . . . . . . of the probability of a one, the number of subjects, and items fixed vs. random, with respect to the data in Table 8 The interaction the number of data in Table The interaction of the probability of a one and subjects, with respect to the 10 . . . . . . . . . . . . . . of the probability of a one and items fixed vs. random, with respect to the data in Table The interaction the number of data in Table The interaction the number of data in Table The interaction 10 . . . . . . . . . . .-. . . of the probability of a one and subjects, with respect to the 12 . . . . . . . . . . . . . of the probability of a one and subjects, with respect to the 17 . . . . . . . . . . . . . . of the probability of a one and items fixed vs. random, with respect to the data in Table The interaction the number of relative power of the test for the subjects by 17 . . . . . . . . . . . . . of the probability of a one and subjects, with respect to the repeated measures interaction based on dichotomous data under Design 2 . . . . . The interaction of the probability of a one and items fixed vs. random, with respect to the relative power of the test for the subjects by repeated measures interaction based on dichotomous data under Design 2 . . . . . viii Page 61 65 77 78 80 9O 91 95 96 10. 11. 12. 13. 14. 15. 16. 17. The interaction of the probability of a one and the level of subject heterogeneity, with respect to the relative power of the test for the subject by repeated measures interaction based on dichotomous data under Design 2 . . . The interaction of the probability of a one and the number of subjects, with respect to the relative power of the test for the subjects by repeated measures interaction based on dichotomous data under Design 3 . . . . . . . . The interaction of the probability of a one and items fixed vs. random, with respect to the relative power of the test for the subjects by repeated measures interaction based on dichotomous data under Design 3 . . . . . . . The interaction of the probability of a one and the level of subject heterogeneity, with respect to the relative power of the test for the subjects by repeated measures interaction based on dichotomous data under Design 3 . . . . The interaction between the probability of a one and the level of subject heterogeneity, with respect to the correlations in Table 21 . . . . The interaction between items fixed vs. random and items crossed vs. nested, with respect to the correlations in Table 21 . . . . . . . . . . The interaction between the number of subjects and items crossed vs. nested, with respect to the correlations in Table 21 . . . . . . . . . . The second order interaction between the proba- bility of a one, items fixed vs. random, and items crossed vs. nested, with respect to the correlations in Table 21 . . . . . . . . . . . . ix 97 99 100 101 108 126 126 127 CHAPTER I INTRODUCTION An experimental design which is often employed in psy- chological and educational research is the repeated measures design.1 The essential characteristic of the repeated measures design is that each subject is evaluated more than once. Thus the simplest of repeated measure designs would represent a situation in which n subjects were evaluated t times, resulting in a data matrix with n rows and t columns. Variations on the simple repeated measures design include assigning subjects to treatment groups and classifying the repeated measures into levels of independent variables asso- ciated with them. In many instances an experimenter will, by necessity, have to evaluate the subjects on each of the t times in a manner such that the evaluations may take on only two values corresponding to, for example, correct or incor- rect responses. When the evaluations can only take on two values (e.g. zero or one), a data matrix of dichotomous values results. Can the familiar analysis of variance, ANOVA, procedure be usefully applied to a data matrix of 1The repeated measures design is sometimes referred to as a split plot design. such dichotomous values? Of the assumptions on which the ANOVA procedure is based, clearly the assumption of nor- mality is violated, and it will be shown that it is likely that the assumption of independence will be violated as well. The problem with which this paper will be concerned is, in general, the analysis of dichotomous repeated measure data, and specifically the applicability of the ANOVA pro- cedure to the analysis of dichotomous repeated measure data. The repeated measures type of experimental design allows not only for the experimental investigation and analysis of events over time, but also offers promise to serve as a vehicle for the investigation cf individual dif- ferential response on the part of the subjects or experi- mental units with respect to the variables of experimental intervention (see Cronbach, Jensen, and others in Gagné, 1967). The importance of determining if it is likely that experimental units have differential responses with respect to the variables of experimental intervention is illustrated in an example referred to by Jensen (1967), of a study by Hovland (1939), who performed an experiment in which no sta- tistically significant differences were found between massed and distributed practice on paired associate learning tasks. After reporting the above mentioned nonsignificance, Hovland went on to report that 44 per cent of the subjects in his study improved more rapidly with distributed practice, 28 per cent learned faster with massed practice, and 28 per cent showed no effect due to the type of practice. The percentages which Hovland reported suggest the possibility of a significant subject by type of practice interaction. A subject by type of practice interaction would indicate that the effect of type of practice was not null but rather different for different types of subjects. In the Hovland study all subjects were measured on learning trials when given massed practice and on learning trials when given distributed practice. Table 1 presents the sources of variation, degrees of freedom, and expected mean squares for data from a study such as Hovland's. Note, there are tests for the type of practice main effects and the trials main effects, but no test for the subjects by type of practice interaction effects. Thus there is no test for what would appear to be an important source of variation in the Hovland data. Examine an experimental design the Hovland study might have employed. In the suggested design subjects are arrayed in two groups (a massed practice group and a distributed practice group) where each subject is given five trials on four randomly selected sets of paired associates. Groups and trials have two and five fixed levels respectively and thus represent fixed sources of variation. Subjects and paired associates are randomly selected from supposedly infinite populations and therefore represent random sources of variation. Table 2 contains the sources of variation, degrees of freedom and expected mean squares for a design such as that Table 1 Sources of variation, degrees of freedom and expected mean squares for data from a design such as Hovland's Source df E(MS) A (subjects) . s-l 2to; B (type practice) 1 tOAB + stag C:B (trials) 2(t—l) GAC:B + soé=B AB 1(s-l) toiB AC:B 2(s-l)(t-l) ozch Note, not all of the above are variances, since there are some fixed effects. Table 2 Sources of variation, degrees of freedom, and expected mean squares for the suggested design Source df E(MS) 2 2 2 2 A (groups) 1 50CS:A + SnoAC + 2008:A + 20noA S:A (subjects: (n-l)2 502 . + 2002. groups) CS.A S.A . 2 2 2 2 B (trials) 4 OBCS:A + 4GBS:A + 2ndBC + 8n0B I 2 2 C (PA 5) 3 50CS:A + lOnGC 2 2 BC 12 CBCS:A + 2nOBC 2 2 2 2 AB 4 GBcs:A + 40mm + noABC + 4nOAB 2 2 AC 3 SOCS A + SnoAC 2 2 ABC 12 GBCS:A + nOABC . _ 2 2 BS.A (n 1)8 OBCS:A + 4GBS:A . _ 2 CS.A (n l)6 50CS:A , _ 2 BCS.A (n 1)24 OBCS:A Note, not all of the above are variances since there are some fixed effects. mentioned in the previous paragraph, given that the assump- tions on which the analysis of varianCe procedure is based, hold. Inspection of the expected mean squares in Table 1 indicates that the ratio, MSBS:A MSBCS:A ' would provide a test for the source of variation--subjects by trials interaction. The source of variation--subjects by trials interaction--would reflect the type of individual differential response to massed or distributed practice that was suggested in the Hovland data, that is the differences between the trial curves for subjects would be greater if there was a significant subjects by type of practice inter- action than would be expected otherwise. Thus had the experimental design suggested in this paper been employed and the ANOVA consistent with it been used to analyze the data obtained, a test for the subjects by trials source of variation would have been available, which if significant would indicate the possibility of a subjects by type of practice interaction. At least at first glance the ANOVA which Table 2 suggests as a possible means of analysis, appears to be a reasonable way to insure a means of testing for a differen- tial response on the part of the two groups2 as well as a means of testing for a subjects by trials interaction. 2By means of a synthetic variance ratio due to Satterthwaite (1941). However, closer examination will indicate violations of the ANOVA assumptions. In order to explain the nature and sources of the violations a digression must be indulged. The digression is necessary in order to discuss population distributions. Consider any population of entities, each entity may have as many as an infinity of attributes and if the number of entities in a population is so few as to be countable in a finite time, a frequency distribution of the entities with respect to any one attribute may be constructed. If one value can represent all of the entities in a population with respect to one attribute, the population can be said to have a point distribution with respect to that attribute. Note that the above same population may not necessarily have a point distribution with respect to some otherattribute. Thus it is important when speaking of the distribution of a population to specify with respect to what attributes the population is distributed. One of the assumptions of an ANOVA is that the response evaluations on dependent variables are independent for each subject or experimental unit. Another assumption of an ANOVA is that the dependent variables have a normal distribution. If the dependent variables have a multivari- ate normal distribution a zero correlation between them is necessary and sufficient to establish their statistical independence. It will now be shown that it is likely that data obtained from a design implied by Table 2 will violate the assumption of independence on which the ANOVA is based. Note in Table 2 that in order to have a test for the source of variation—~subjects by trials interaction--it is neces- sary to regard the sets of paired associates as a random sample of Sets of paired associates sampled from some popu- lation of sets of paired associates. Had the sets of paired associates been regarded as all of the sets of interest, "sets" would not have provided a random source of variation and the Table of sources of variation, degrees of freedom and expected mean Squares would have been as in Table 3. An inspection of Table 3 indicates no test for the source-- subjects by trials interaction. If the source of variation, sets, is random, it is unlikely that the effect on the dependent variable would be null. In order for the effect of sets to be null when sets are random the distribution of sets would have to be a point distribution with respect to the attribute-effect on the dependent variable. If the effect of sets is non-null it is likely that there will be a positive correlation between subjects across sets. Simi- larly, if the effect of subjects within groups is non-null, which is very likely, it is likely that there will be a positive correlation between sets across subjects. Why is it likely that there would be a positive corre- lation between sets, for example, if the effect of subjects on the dependent variable is non-null? The reason can be stated generally in terms of experimental design. In the Table 3 Sources of variation, degrees of freedom, and expected mean squares for the suggested design when paired associates are fixed Source df E(MS) A (groups) 1 200S:A + 20nd: S:A (subjects:groups) (n-l) 200S:A B (trials) 4 4OBS:A + 8n0é c (PA's) 3 SOéS:A + lOnoé BC 12 OBCS:A + 2no§c AB 4 4OBS:A + 4noiB AC 3 50CS:A + Snoic ABC 12 OBCS:A + nogBC BS:A (n—l)8 4OBS:A CS:A ‘ (n-1)6 SoészA BCS:A (n-l)24 OBCS:A Note, not all of the above are variances, since there are some fixed effects. 10 absence of a disordinal interaction between two independent variables in an experimental design whose levels are com— pletely "crossed," a non—null effect of one independent variable, say A, will result in positive correlation between the levels of the other independent variable, say B, across the levels of the non-null independent variable, A. If the previous sentence is not intuitively sufficient, consider the following example. Let A and B be two independent vari~ ables whose levels are completely crossed. Allow the variance of A, B and error to be defined as, CA2 > 0, CB2 = 0, and oez > 0, respectively. If A and B are the only independent variables, then any observation is a function of the effect of a level of A and error. Then the covariance of two levels of B, for example level one and level two, across the levels of A will be equal to E[(A+el)(A+e2)] - E[A+e1] E[A+e2], which is equal to the variance of A plus the covariances of A and e1, A and e2, and e1 and e1. If the preceding three covariances are assumed to be equal to zero, the covariance of two levels of B is equal to the variance of A. Since 0A2 > 0 and the variances of the errors is greater than zero the correlation will be greater than zero. In case the reader is not familiar with what is meant by the term crossed, note that crossed as used above means that all the levels of factor A occur in combination with all of the levels of factor B so that the number of combina- tions is equal to the Cartesian product of the number of 11 levels of A and the number of levels of B. The term crossed is used in contradistinction to nested which implies a two stage selection procedure. Where, if the levels of A were nested within the levels of B, the levels of B would be selected first and the levels of A could then be freely selected (without replacement) within each of the levels of B. As a matter of definition, the levels of the factor which represents the subjects or experimental units are crossed with the levels of the factor which represents repeated measures in a repeated measures experimental type of design. Recognize at this point that all "within" type factors have levels crossed with or nested within the levels of the factor for repeated measures and have levels which are crossed with the levels of the factor which represent the subjects or experimental units. Thus in order to not violate the assumption of independence between the response evaluations for subjects or the experimental units the factor for repeated measures must not represent a non-null source of variation nor may any factor with levels crossed with or nested within the levels of the factor for repeated measures. As previously mentioned only in the trivial case of a point distribution would a random source of variation be a null source of variation. If a factor does not represent a random source of variation, no generalization can be made to the levels of that factor which did not occur in a given experiment. This is a severe limitation, in that items are 12 almost always crossed with or nested within repeated measures and experimenters seldom wish only to make a con- clusion that is restricted to the items which they actually use in an experiment. Next consider the type of response which must be evaluated in an experiment designed to be consistent with Table 1. In a paired associate task a stimulus "word" is paired with a response "word," by the experimenter, for the subject, in the initial phase of the experiment. There— after when presented with the stimulus word a subject is to respond with the response word, which the experimenter has indicated should be associated with the stimulus word. It would often be very difficult for an experimenter to evalu- ate a response in any other than a dichotomous fashion. That is the subject either recalled correct response word or the subject did not. Next consider the number of paired associates in a set. If the number is as small as two the evaluation of the set would be a discrete variable which could take on the values 0, l, or 2, corresponding to both incorrect, one correct and both correct respectively. Thus the dependent variable would be a discrete variable with a three point distribution rather than a normal random variable with a continuous normal variate as assumed by the ANOVA. In order to circumvent the problem associated with the random non—null source of variation, items, crossed with the source of variation due to subjects or experimental units, a 13 typical experimenter will sum the response evaluations for the stimuli to form a repeated measures score. He will then do his analysis on those repeated measures scores. In doing so, he has eliminated one problem, but caused another. In order to demonstrate this problem three repeated measures experimental designs will be defined and the analysis for them will be discussed when stimuli provide a random source of variation. It will be shown that when the response evaluations are simply summed and ignored as a factor in the design, the test within the ANOVA framework for a repeated measures effect is not correct. The first of the three designs is the simplest form of a repeated measures experi— mental design. It has only two factors: subjects and repeated measures. For the purposes of this paper this simple design will be called Design 1. If the repeated measures scores in Design 1 are formed as the sums of response evaluations to items, two more designs can be con- sidered. The first of these two will be called Design 2 in which the factor items is crossed with the factor repeated measures. In the second which will be called Design 3, the levels of the factor items are nested within levels of the factor repeated measures. Design 2 may be termed a three way factorial design with subjects crossed with repeated measures and items. Design 3 is a three way factorial with subjects crossed with repeated measures and items and items are nested within repeated measures. Tables 4, 5, and 6 represent 14 Table 4 Sources of variation, degrees of freedom, and expected mean squares for Design 1 Source df E(MS) - _ 2 2 A (subjects) s l 0e + roA _ 2 2 2 B (repeated measures) r 1 Ge + GAB + soB _ _ 2 2 AB (5 l)(r l) 0e + CAB Note, not all of the above are variances, since there are some fixed effects. Table 5 Sources of variation, degrees of freedom, and expected mean squares for Design 2 Source df E(MS) ‘ _ 2 2 2 A (subjects) s 1 Ge + rOAC + troA B (repeated r-1 02 + 02 + to2 + so2 + sto2 measures) e ABC AB BC B _ _ 2 2 2 AB (5 l)(r 1) Ge + OABC + tOAB . _ 2 2 2 C (items) t 1 Ge + rOAC + rsoC _ _ 2 2 AC (5 l)(t 1) 0e + rOAC _ _ 2 2 2 BC (r l)(t 1) 0e + OABC + SOBC _ _ _ 2 2 ABC (5 l)(r l)(t 1) Ge + OABC Note, not all of the above are variances, since there are some fixed effects. 15 Table 6 Sources of variation, degrees of freedom, and expected mean squares for Design 3 Source df E(MS) . _ 2 2 2 A (subjects) 5 l 0e +OAC:B + troA B (repeated r—1 02 + 02 . + to2 + so + so2 measures) e AC B AB C.B B _ _ 2 2 AB (8 l)(r 1) 0e + OAC:B + toAB . - _ 2 2 C-B (items) (t l)r 0e + GAC:B + soc B . _ _ 2 2 AC.B (s l)(t l)r 0e + OAC:B Note, not all of the above are variances, since there are some fixed effects. respectively for Designs 1, 2, and 3 their sources of varia- tion, degrees of freedom, and expected mean squares. For Design 1, it can be seen by inspection of expected mean squares that a test statistic may be formed as the ratio MSB MSAB (1.1) to test the effect due to repeated measures. The computa- tional formulas for these two mean squares with respect to Design 1 are r S 2 r S 2 z ( 2 xi" ( z z xJL ) j=1 1:1 3 j=l i=1 3 S rs MS = B r - l l6 r S r S 2 2 xi. 2 z ( 2 xi.)2 j=1 i=1 3 _ j=1 i=1 3 _ 1 s MSAB ' (r - 1)Ts - 1) S r r S 2 ( z X..)2 ( z z X..)2 i=1 j=1 13 + j=1 i=1 13 r rs (r - l)(s - 1) where Xij is an observation for repeated measure j on sub~ ject i, j = l, 2, ... r, i = l, 2, ... s. Noting the above formulas examine the computational formulas for the same mean squares with respect to Designs 2 and 3 (the formulas are identical for Designs 2 and 3). These formulas are r s t 2 r s t 2 Z ( Z X X.. ) ( Z Z Z X.. ) j=1 i=1 k=l 13k _ j=1 i=1 k=l 13k _ st rst MSB — r - 1 and r s t 2 r s t 2 Z Z ( Z X.. ) X ( 2 Z X.. ) j=1 i=1 k=l 13k - j=1 i=1 kzl 13k ' _ t st MSAB I (r - l)(s - l) s r t r s t z ( z z x..k)2 ( z 2 2 x..k)2 i=1 j=1 k=1 13 j;1 i=1 k=l 13 rt rst (r- l)(s - l) where Xijk is an observation on subject i for repeated measures j and item k. i = l, 2, ... s, j = l, 2, ... r, and k = l, 2, ... t. By inspection of the two sets of computational formulas, it can be observed that the mean squares MSb and MSAB for Design 1 are linear transformations of the same mean squares for Designs 2 and 3, that is of 17 course given that the formulas are applied to the same data such as the Xijk observations defined above and given that the repeated measures scores in the case of Design 1 were formed as a simple sum of response evaluations to the items which are associated with the repeated measures. Since the linear transformation for each mean square was the same, ratio 1.1 will be invariant across Designs 1, 2, and 3. The invariance of ratio 1.1 provides interesting implications for the case in which items are a random non- null source of variation, and yet Design 1 was considered appropriate and the analysis consistent with it was employed. Inspecting the expected mean squares for the source--repeated measures and the source--subjects by repeated measures interaction in Tables 5 and 6, it can be seen that ratio 1.1 is not the appropriate statistic to test for the effect due to repeated measures. Therefore, it should be apparent that if repeated measures scores are formed as a simple sum of response evaluations for items, the analysis which is implied by Design 1 is inappropriate. If the expected values for the mean squares are substituted in ratio 1.1 for Designs 2 and 3 the kind of errors which can arise becomes apparent. For Design 2 the substitution results in the ratio 2 2 2 2 2 Zoe + OABC + tGAB + SOBC + stoB 2 2 2 + Zoe OABC + tOAB 18 For Design 3 the substitution produces + 502 + sto2 2 2 2 20e + OAC:B + toAB C:B B 20: + CAC:B + togB Thus, when ratio 1.1 is employed to test for an effect due to repeated measures an implicit assumption has been made. If Design 2 is appropriate the assumption is that CBC = 0. If Design 3 is appropriate the assumption is that OC:B = 0. If the implicit assumptions are not valid it is possible to obtain a spuriously high value of ratio 1.1. Because of the 2 2 BC C:B to zero it appears important to include items as a factor in possibility that o is unequal to zero or 0 is unequal the design. When items are random, if the factor for items is included in a repeated measures experimental design, a result due to Satterthwaite (1941) provides that quasi-F ratios may be formed to test hypotheses concerning an effect due to repeated measures. To illustrate, if items are MSB + MSABC MSAB'+ MSBc will crossed with repeated measures the ratio have a sampling distribution that approximates a central F distribution, when the effect of B is null. If on the other hand, items are nested within repeated measures the ratio MSB + MSAC:B MSAB + MSC:B has a sampling distribution which approximates a central F distribution, for a null situation with respect to the effect of B. Hudson and Krutchkoff (1968) did a Monte Carlo study of the empirical probability of a Type I error and the empirical power of the quasi-F test based on 19 normal variates and concluded that the quasi-F had properties that were generally Similar to the F—test. When dependent variables are formed as the sum of response evaluations to several items, an approximation to normality may be pleaded on the basis of the central limit theorem. But, when the response evaluations to items them- selves are the dependent measures it is then the response evaluations which must be distributed normally in order to meet the assumption. In many instances a response to an item is evaluated as either a one or a zero corresponding to either an acceptable response or one which is unacceptable, in which case the dependent measure would have a two point discrete Bernoulli distribution rather than a continuous normal distribution. The problem has now been shown. The experimenter who would like to regard the items he employs in an experiment as a random sample from a population of items which has other than a point distribution with respect to the attribute of the items which effects the response evaluations, who sometimes must evaluate responses as zeros or ones, and who would like to employ the analysis of variance as a means of analysis is in a difficult position. If he fails to include items as a factor in his design one test in the ANOVA will be questionable. If he includes items as a factor he has two violations of the ANOVA model with which to concern himself. Thus it would appear that an attempt at a resolution of the above dilemma is justified. Two possible means of 20 dealing with the dilemma occurred to this investigator. The first would ignore the ANOVA and attempt to formulate a new model and analysis to fit the situation and Chapter II will follow this line of investigation. The second would be to demonstrate that the ANOVA is robust to violations of inde- pendence and normality and Chapters III, IV, V and VI will examine this possibility. CHAPTER II MODEL DEVELOPMENT A series of models are developed in this chapter. The models are developed for two reasons. The main reason was to get a better understanding of the experimental situation and the insight necessary to develop the data simulation algorithms discussed later. The decisions which were made with respect to the techniques of data simulation, as out- lined in Chapter IV, were directly influenced by the discussion and development presented in this chapter. The secondary reason was to examine the models with the hope that a method of data analysis would become apparent which had the advantages that an ANOVA would have when the ANOVA assumptions are met, but which did not require assumptions of independence and normality. The first model to be developed in this chapter will represent the response evaluation of the response of a random subject to a random stimulus when the evaluation can only take on the values zero be an extension of the first evaluations of the responses of random stimuli. Then the or one. The second model will so as to represent the response of a random subject to a series model will be extended to 21 22 represent possible experimental intervention between any two stimuli in the series which are presented to a subject. Finally a model will be considered to represent the situa- tion in the previous sentence for an arbitrary number of subjects. To set the background for the first model attention must be given to specification of the nature of the popula- tion of subjects and of the population of stimuli. First consider the population of subjects. Each of the subjects in the population will be allowed infinitely many specific abilities. All subjects will be allowed all abilities albeit each in varying amounts. Thus the subjects in the population may be distributed with respect to each of infinitely many abilities associated with subjects and the distribution of the population with respect to each will have some density. Therefore, if ability u is of interest in any particular situation the subjects can be considered as distributed in the population with respect to u and have density fu‘ Now consider the population of stimuli. Each stimulus may be considered to have a potential with respect to eliciting a response within the domain of responses in which the experimenter is interested. The potential that a stimulus has to elicit a response within the domain of responses in which an experimenter is interested is in part conditioned on the understanding a subject has with respect to what is expected of him. The understanding or expecta— tion that a subject has is a function of the instructions 23 which the experimenter provides and the atmosphere of the experimental situation. For example, if a subject, because of the instructions an experimenter gives him, understands that it is his ability with respect to u which is to be brought to bear in responding to say stimulus A, the poten- tial of A may be different than if the subject thought that an ability other than u should be brought to bear in respond— ing. A more concrete example would be represented by a situation in which the stimuli are words. If the subject thinks he is to give a free associate to, for example, the word girl when the experimenter wants a definition, the response potential of the stimulus word girl might be differ- ent with respect to the experimenter's criterion of an acceptable response than if the subject thought he should give a definition. Thus the stimuli in the population of stimuli may be distributed in infinitely many ways each with respect to the understanding respondents have about which of their infinitely many abilities should be brought to bear in responding. Let the respondent and the experimenter both understand that it is ability u which is to be brought to bear in responding to all stimuli. And, let p be the response potential of a stimulus with respect to ability u. Stimuli in the population of stimuli may be distributed with respect to p and have a density fp. One further parameter may effect a response evaluation. Since, given a subject, a stimulus, and a joint understand- ing between experimenter and subject the response evaluation 24 is not determined. The remaining parameter is the criterion the experimenter has with respect to what is an acceptable response. Let the experimenter's criterion be a constant c for a given experiment. The background concepts for modeling have now been established. Before the first model is developed, however, it will be useful to establish some conventions. For the balance of this chapter unless otherwise noted capital Roman letters will be used to represent variates.1 The corre- sponding lower case Roman letters will represent an evalua- tion or obtained value (a constant) of that variate. ¢(u) will represent the standard normal distribution function evaluated at u. ¢(u) will represent the standard normal density at u. Fv(x) is the cumulative distribution function of the variate V evaluated at x; and fD(y) is the density of the variate D at y. With these conventions established the modeling of the response evaluations may begin. Let the variate U represent a random subject, then u is the ability of a given subject which has been sampled. Let P represent a random stimulus and p a given one. Then let b represent the response evaluation of the response of subject u to stimulus p made by an experimenter with criterion c. The response evaluation b may be defined as a 1The term variate will be considered synonymous with random variable, as it is in much of the statistical literature. 2That is the cumulative up to u. 25 function of the three values u, p, and c, conditioned on no error in the evaluation and no misunderstanding of the subject. That is b = f1(u,p,C)- Analogously a random response evaluation may be defined as B = f1(U,P,C) where B is a variate which may take on only two values (zero or one). B is thus a Bernoulli variate and has the familiar probability function _ b _ I'b PB(b) — e (1 e) where 8 is the probability that B = l. The value p has been defined as the potential that a particular stimulus has with respect to eliciting an accept— able response. It will be useful to define a value g', where g' = l-p. Let the range of g' be specified as 0 i g' i l, where if g' = 0 no matter what u and c are, b will equal one and where if g' = 1 no matter what u and c are, b will equal zero. Thus 9' is in some sense the poten- tial difficulty of a stimulus, with respect to an acceptable response. Let the range of u be -w < u < w and let the density of u be ¢(u). The quantile rank, r, of u is given by r = @(u), and has a uniform density on the interval (0,1). Note that r is the probability that the variate U is less than the value u, or in symbols, @(u) = Pr(U : u), where -w < u < w . Note that the inverse exists as well, u = ®-l(r), 0 i r i 1. With the above values defined the function which will 26 represent one response evaluation, f1, may be defined b = f (r,g',c) = {0' 1 l. The value c is really a part of the definition of an accept- able response, and so may be incorporated into the value which represents the stimulus' difficulty. So let g=g'+c, then b = f1(r,g) = { i E 3 Observe that f1 is an expression of conditional probability if the variates corresponding to b, r, and g are substituted in the expression. Assuming that R and G are stocastically independent the conditional probabilities are Pr(B = OlG = g) = Pr(R : g) = FR(g) = g Pr(B = 1|G = g) = 1_Pr(R : g) = 1—FR(g) = 1-g from which it is apparent that -b or more simply written in the shorthand notation The joint probability density of B and G is PB'G(b,g) = P (blg) fG(g) BIG and the probability function of B is then 3Because r has a uniform probability density. 27 _ 1 PB(b) - Of PB,G(b’g)dg It will be possible to complete the development of the first model once fG(g) is specified. We will assume that fG(g) belongs to a family of Beta distributions (this family will be large enough to contain nearly all kinds of distributions for G that are encountered in practice). Thus we assume f 95(1-g>,03g_<_1, _ (s+f+l)l fG(g) - —-—T;T—— S. where the parameters 5 and f are non-negative integers. Then I 1 + + . — s.fl or I 8+1 1) = 0 P (b) = (s+f+l). fl gs+1—b (l—g)b+fdg = s+f+2 B I I 0 f+l b _ l s.f. s+f+2’ - which models the response evaluation of the response of one random subject to one random stimulus. Observe that the expression f1 which was shown capable of representing a probability conditioned on g may represent a probability conditioned on r as well. That is the P (B = 1|R = r) = FG(r) and the Pr(B = OIR = r) = l-FG(r), from which the conditional probability function PBIR(bIr) = (FG(r))b (l—FG(r))1_b 28 may be written. Let g represent a vector variate FG , G , ..., G 1', where the G., i = l, 2, ..., n, are 1 2 n 1 representative of n stimuli sampled independently from the n population of stimuli. Thus FG(g) = H G (gi). Let B — i=1 i represent a vector variate fBl, B , ..., Bn1'. Then the 2 conditional probability function for B given R may be written as n b- - . (9|r) = n (F (r)) 1 <1—FG (r))1 b1 i=1 i ' PEIR 1 G and since fR(r) = l, 0 i r i l , the joint probability density function for B and r is P§,R(b,r) = PEIr(bIr) and the probability function for B is (g) = ofl "3:5 (FG_(r)>bi<1—FG_ (9)) — Pr(G _<_g) — FG(g) (Wadsworth and Bryan, 1960), from which we may write va = Pr(G _<_ (v)) = FG(<1>(v)) or _ (V) FV(V) — of fG(t)dt Then taking the derivative of both sides 30 fV(V) = fG((v)) d((v)) , by the chain rule and thus fv(v) = fG(¢(v)) ¢(v) Recall at this point that fG(p) is a member of the Beta family of distributions, that is = (s+f+l)! fG(¢(v)) (¢(v))s (l-¢(v))f s.f. Let n = s+f, then f = n—s thus we may write I _ fV(v) = _leilll_,(¢(v),s <1— (v-1)1(m-v). We observe that fV is the density of the yth order statistic of a sample of size M sampled from a population with standard normal distribution. Values of the first two moments of fV have been given by Tiechroew (1956) for small values of y and m. Recall that the previous modeling was based on the function f1 in such a manner so as to obtain a probability statement conditioned on g and r. Now that it is wished to account for a systematic change in u, it will be helpful to define a function which will lead to a probability condi- tioned on u. Before the new function is defined, note that since ¢'lis a strictly monotone function, that 31 g 1 r «=+4 @“1(g) : 2-1(r) Therefore the function f2 defined as b f2 (¢'1(r), ®'l(g')), 0, ¢'l(g) > ©‘1(r) 1, ¢-‘(g) : u = { l, v < u, is strictly analogous to the function f . Again the condi— 1 tional probabilities may be written, w E? n O c u c n H I w < /\ c n l-Fv(u) and "d ‘6 ll ...: C.‘ II C II Pr(V i u) fv(u) OI.” PB|U(b|u) = (Fv(u))b (1-Fv(u))1'b. The joint probability density of B and U is PB,U(b’u) = (FV(u))b <1—Fv)1'b ¢(u) and the probability function of B is PB(b) = _mfm(FV(u))b (1—Fv(u))1‘b ¢(u) du For a series of n stimuli sampled independently n FV(u) = E F (u) and 4If and only if. 32 132(9) = 00 oo 00 n b. 1b 1 _ _ . _mf mf . ~_mf i:l(FVi(ui)) (l Fvi(ui)) 1¢(ui)du1du2...dun. Note that _ u- FVi(ui) — _£ 1 fv(t) dt or _ ui Y-I _ m‘Y Fvi(ui) — K _wf (¢ (1 ©(vi)) ¢ 0, yijk = 1 ' Following the example of Lunney (1969) the probability of a one in [y. ] was either .5, .2, or .1 (given all null ijk effects) for which h took on the values of 0, .84, or 1.28 respectively. Recall from Chapter II that the Beta family of vari- ates may represent most of the distributions of random item difficulties that are encountered in practice. Also recall the relationship between a Beta variate and an Order variate, that is, that an item effect for normal data which is Order (y, m) can produce an item effect for the same data after dichotomization, that is, Beta (y-l, m-y).2 In prac- tice, this investigator has observed that tests which have mean item difficulties of .5, .2 and .1 often have a dis- tribution of item difficulties which may be approximated by the densities of Beta (9,9), Beta (2,10) and Beta (1,14) 2Recall from Chapter II that s = y—l and f = m—s—Z. 51 respectively. The corresponding order variates would have densities of Order (10,19), Order (3,13) and Order (2,16) and would have variances .0808, .1514 and .1739 respectively, the variances employed in this study. In review, first subject heterogeneity was taken into account by the algorithm and four levels were employed. Next, items were allowed to be fixed and null in effect or random and non—null in effect and crossed or nested. Then, the null or non-null effects of repeated measures and the null or non-null effects of subject by repeated measures interaction were allowed. Following the above, the normal data were dichotomized and three levels of a probability of a one in [y. ] were employed. ijk In this study interest did not center on the effect of varying the number of items or on the effect of varying the number of repeated measures; thus r and t were set at four and three respectively, values which were as small as would usually be found in practice. Five levels of s were inves— tigated beginning with s = 4, a very small value, and then s = 6, 8, 10 and 12: values which were not unusual in practice. Three sets of expected repeated measure mean scores for the dichotomous data were selected for the non- null repeated measure effect cases. The means were: (1) .75, .50, .50, .50; (2) .36, .20, .20, .20; and (3) .19, .10, .10, .10. Note that since for a Bernoulli variate the variance is p(l—p) where p is the probability of a one, that for h = 0, .84 and 1.28 the expected variances are .25, .16 52 and .09 respectively. Then note that for the non—null repeated measures cases defined on the dichotomized data the first expected repeated measure mean exceeded the others by the amount of the null case variance. Thus the non— centrality parameter given by the expression where B. is the effect of the jth repeated measure, will be the same for all h. For the non-null situation with respect to the inter- action the same values were used to deviate the repeated measure scores from the rest as were used for the repeated measure main effect non-null cases. The first half of the subjects' scores were deviated in one direction and the other half were deviated in the other direction producing a null effect for repeated measures main effect, but a non- null interaction effect for which the non-centrality para- meter should be the same on the dichotomous data for all values of h. Once both [Xijk] and [y. ] were in their final con- ijk figuration an analysis of variance was performed on both. The values of the statistics obtained from each ANOVA were then used to increment various sums, sums of squares, sums of products, and counters associated with critical values of the statistics in question. Then the entire generation process was repeated 999 more times so as to result in the 53 generation of 1000 samples for each case of interest. Once 1000 samples had been generated the correlation between all mean squares for the dichotomous data were calculated and printed. Then for the statistics which were of interest the frequency of values larger than a.1000 were printed for q = .05, .025 and .01. Four levels of subject heterogeneity, three levels of a probability of a one, five levels of the number of sub- jects, items fixed or random, crossed or nested and one null and two non-null situations resulted in a 4x3x5x2x2x3 array of 720 cases for which data were generated. The results of those generations will be presented in the next chapter. The entire list of the generation program may be seen in Appendix C. CHAPTER V RESULTS OF THE MONTE CARLO SIMULATIONS In this chapter data will be presented with respect to the fit of F-distributions to the empirical sampling dis- tributions of variance ratio test statistics for tests of repeated measures main effects1 and subjects by repeated measures interaction effects, under all of the conditions and parameter configurations outlined in Chapter IV. For each condition and parameter configuration the data will consist, in part, of the frequencies of 1000 test statistics based on 1000 simulated data samples which had values occurring in a = .05, .025, and .01 rejection regions.2 The presentation of the empirical correlations between the mean squares in the variance ratio tests for repeated measures main effects and the subjects by repeated measures inter- action effects will complete the presentation of raw data. For each data table the significant trends within the table will be indicated, significant interactions graphed and lBoth for the "ordinary" variance ratio tests and for the "quasi-F" variance ratio tests. 2The rejection regions being defined by an F- distribution with the same degrees of freedom as the variance ratio test concerned. 54 55 summary mean statistics reported where appropriate. Because the correlations between the frequencies in a = .05, .025 and .01 rejection regions across all cases in each table of frequencies were generally greater than .9, detailed summary mean statistics will only be reported for frequencies in the a = .05 rejection region. Similarly significant interactions will only be graphed with respect to frequencies in a = .05 rejection regions. The purpose of the Monte Carlo simulations discussed in Chapter IV was to determine if it was likely that ANOVA procedures could be "appropriately" employed for the analysis of data such as was simulated for this study. In order to give a reasonable consideration to the results of the simulations, it is necessary to have some criterion for "appropriate employment" of ANOVA procedures. For the purposes of this paper "appropriate employment" will be adjudged in terms of hypothesis testing and two conditions will be considered as necessary and sufficient for it. The first of the above two conditions requires that the empirical probability of a type I error, for a given hypothesis testing situation, is "reasonably close" to the nominal probability of a type I error as determined by ANOVA considerations. The second of the above two conditions requires that the power of a test of a true non-null hypo— thesis is not "unreasonably small." The phrases "reasonably close" and "unreasonably small" will be discussed below. If 1000 samples are simulated so that a null hypothesis 56 is true and so that the assumptions of an ANOVA are met, the number of F-statistics testing the above hypothesis which have values which exceed the Fl—a quantile of a correspond- ing F-distribution3 will be approximately 1000a. If 1000 samples of data are simulated so that a null hypothesis is not true and so that the assumptions of an ANOVA are met, the number of F-statistics testing the above hypothesis which have values which exceed the F quantile of a l-a corresponding F-distribution will be approximately 1000 times the nominal power (1-8) for the situation simulated. Let Xi be defined according to the rule 1' Fi > Fl-q 0, otherwise Xi = th where Fi is the F-variate calculated on the 1 sample, i = l, 2, ..., 1000, and F is the l-a quantile of the l-d corresponding F—distribution. The variate Xi is then an indicator variable, which takes on the value one when Fi is in the a rejection region and which takes on the value zero when Fi does not occur in the rejection region. Note that the frequency of Fi's which fall in the a rejection region, 1000 F, may be represented by the expression f = 2 xi. i=1 Observe that f is a binomial variate with parameters p = a and n = 1000. The variance of f is then np(1-p) or for n = 1000 and a = .05, the var (f) = 47.5. Therefore a .95 3An F-distribution with the same degrees of freedom as the variance ratio for the test. 57 probability interval may be formed about the expected value of f, E(f) = 1000 a, which for a = .05 is Pr(36.5 < f < 63.5) = .95. We now have a basis on which to define what "reason- ably close" to the nominal probability of a type I error may be. If the empirical frequency of 1000 variance ratio test statistics, testing a true null hypothesis, which fall in an a rejection region as defined by ANOVA considerations, is a frequency which occurs in a .95 probability interval about 1000a, the empirical probability of a type I error may be considered reasonably close to the nominal a. The problem of what an unreasonably small value of empirical power would be is more difficult to resolve than the question concerning the empirical probability of a type I error which was considered above. Clearly an empirical power less than or equal to a would be unreason- ably small (a test with this property is sometimes called a biased test), but beyond that clear lower limit, the matter must in this investigator's judgment, be arbitrary. For the purposes of this paper it was arbitrarily decided that an empirical power of less than one third the nominal power would be unreasonably small. Table 7 contains frequencies in a = .05, .025 and .01 rejection regions of variance ratio tests of repeated measure effects under conditions in which the repeated measure effects were null, items were crossed with repeated measures and there was no interaction between subjects and 58 .. .. ...N 3 on ..e N e 2 NN n. 3.. N ..N 3.. NN 3 e. .. n NN NN AN NN 9. 3 NN ..N NN ..N 3 3 NN N. N on NN N N 3 N ..N N.. on ...: N 2. 3. 3 NN e .. NN 3 3 .... N :85 ... 3 N N 9. NN ..N NN NN N 3 N.. 3 N NN 9N Nm om N ... .. ..N .. om NN m e N NN 0.. on NN 0N NN ..N Nm 8 . NN NN n on. ..N 3.. 3 NN .. ON 3 on NN n N NN NN 3 3 N .. N .3 3 N. NN .. N N N N... S 3 3 N N No 3 N eon: N. .. 3 3 NN 3 N N N NN No .... NN NN NN N No 3 N NN o «N o N.. NN ..N N 0.. 3 an 2 NN NN NN ...N on Na .. o 0 ON N 3 NN NN e NN NN ..o N“ NN N .N «N 3 NR N 3 N N NN N.. 3 4 3 N N N No N.. NN NN N o... 3. NN N :83 3 NN NN NN an N.. N. N NN 3 .... NN NN 2 0.. NN oe NN N NN N 3 N 3 ON 3 .. N 3 NN ...... NN N N 3 Nm NN .. oN NN o NN N NN. N ..N N ..N 3 Ne .. 3 N NN NN No 3 N e o ...,N 3 NN N.. NN .. N 3 on T. 3 N NN 3 9. N.. N v2.2 3 NN NN NN N... N 3 N 3 NN 9.. SN 3 o N 3 3 .... N n N NN n N. NN N o NN NN .... N N NN N ON 3 NN .. .. N NN N. ..N ....N N N NN N NN 3 e S NN 0.. NN 3 N e .. N 3 ..N 3 N N N 3 N.. NN o N N 3 No N.. N 9253 N a NN ...N 9. N ,.. N ..N 3 ..N NN NN 3 ON 3 NN. NN N N e ..N e N. 3 e 0 ON NN .... N S ..N NN 3. .... Ne .. N 3 3 N n 3 0.. e 2 NN NN N.. N 3 ..N 3 N N.. am N .. .3 3N NN f. 3 .. 3 N 3 N.. N.. o . N NN NN N N.. N 3.5 .. N 3 3 N NN .. N N 3 3 ..N N S 3 ON 3 N. N N o .. NN NN NN N N NN N N N o N 2 3 NN NN .. .. N NN o ..N N 3 e S NN NN o o a 3 3 ..N NN N .. N NN ... N 3 N m 3 NN eN ON 3 e NN NN N... e. N .825 N N N ..N NN NN N 3 ..N NN RN 3 NN N NN NN 3 m... N n N. NN e N 3 N N 3 NN NN oN NN ..N NN NN NN. No .. e n. o ..N N N NN N 3 NN N ..N N ..N 3 NN ..N NN NN N N o NN o ..N NN o e ..N 3 NN on o o NN NN on on N 3me N N NN 3 3 N4 0 N 3 3 NN a e N NN N 0.. NN N N a NN .. 0.. NN NN o ..N e 0.. ON S N 3 3 f. NN .. o N ..N N 3 N ..N N N NN NN NN NN e S 3 on NN N ... 4 3 3 .. 3 3 .. 3 N .... N ..N N NN 0N NN 0.. N See; 3 N 3 ..N .... 3 2 N ON a 3 NN NN N N N 3 N.. N N.. n 3 N N... N. ..N N NN e 3 NN _ 3 N NN NN N. T. .. . .. N 4 N4 N on NN 3 N NN e on ..N 3 .. NN 3 .... 3 N ..N .. N N NN 3 o N NN a NN NN o o NN NN 3. N.. N can: 2 N N.. 3 No N.. NN NN ..N NN N.. N.. N N ON 0N .. NN N 53 .....N 253 .33... no N035 mo .02 z a z a z a z a z e z e z a z e z e 133 No 38823 2 NN on 3 NN a... 2 N on 82 . n N. N. N. 3o a. g N . :3: .33 3033 95:0: voyages 33 253033.» .3395 253 .33 39:30:03 can Nae: co 60.3.3503: v3.2.3 now u on» «a .2530: 6360?: 83... 5 .3353on 13:25 05. .N 033. 59 repeated measures. Table 7 is laid out as a six—dimensional array with three left margins and three upper margins. The left-most margin indicates the number of subjects employed with respect to a given simulation of data. Proceeding from left to right the next margin indicates the nature of the items, whether fixed and null in effect or random and non— null, and the right-most of the left margins indicates the level of subject heterogeneity. The upper margins from top to bottom indicate: (1) the probability of a one with respect to a given simulation of data, (2) one thousand times nominal a, (3) an indication of whether the 1000 vari- ance ratios with respect to a given cell were calculated from the dependent variables when they were variates with normal density (N) or from the subsequently dichotomized normals (D). In order to test for trends in the data reported in Table 7, the margins were considered as fixed sources of variation and a multivariate analysis of variance was per- formed on the frequency data three-tuples4 within the table, employing the highest order interaction mean products as an estimate of error under the assumption that the highest order interaction is truly null. From the first analysis of the data in Table 7 it was concluded that the frequencies for tests based on the dichotomous variates with overall mean vector 39.9, 18.5, 4For a = .05, .025, and .01. 60 6.75 were significantly different from the frequencies for tests based on the normal variates with overall mean vector 46.9, 26.4, 9.7. Then subsequent analyses were performed on frequencies with respect to dichotomous and normal variates separately. With respect to the frequencies based on dichotomous variates, it was concluded that there were significant main effects due to the probability of a one and the number of subjects as well as an interaction between the two signifi— cant main effects. The significant interaction is repre- sented in Figure 1. In the figure the two horizontal lines represent .95 probability limits for mean frequencies such as those graphed, given an expected value of 50. Observing Figure 1, it appears that a favorable comparison of nominal a and empirical probabilities of a Type I error occurred when the probability of a one was .5 and there were six or more subjects. Also a favorable comparison occurred when the probability of a one was .2 and there were ten or more subjects. However when the probability of a one was .1 no favorable comparisons occurred, although the graph suggests that a favorable comparison might occur given more subjects. The marginal mean frequencies for the a = .05 regions for probabilities of a one equal to .5, .2, and .l were 50.9, 39.9, and 28.8 respectively, and for numbers of subjects equal to 4, 6, 8, 10, and 12 they were 27.8, 39.2, 40.9, 5Ordered for frequencies in a = .05, .025, and .01 rejection regions, respectively. 61 GOr 50— U) c o w m m m a __n r - O \ -H 4.) 8 .3 4()_. m m In C u c 'H Number of Subjects 8 30"» _ .8 I2 - C 5 |O=O '85 3' 8=' m 6‘: D 4i=‘* 2C)h- J c) l l 1 C15 C12 (1| Probability of a One Figure 1. The interaction of the probability of a one and the number of subjects, with respect to the data in Table 7. 62 42.0, and 49.7 respectively. With respect to the frequencies based on normal variates there was an unexpected effect due to the number of subjects. The marginal mean frequencies for a = .05 regions were 48.2, 34.4, 43.5, 55.1 and 53.0 for numbers of subjects 4, 6, 8, 10 and 12 respectively, where the mean frequency for six subjects differs significantly from the other mean frequencies. The only reason this investigator can give for the unexpected low mean frequency for six subjects is that it was a peculiar occurrence which would not reoccur if the simulations were repeated with a different starting value for the random number generator. Table 8 is analogous to Table 7 in that it differs from Table 7 in only two aspects: (1) the values in the table were obtained by simulating data which could have arisen from Design 2 rather than Design 3, and (2) no fre- quencies appear with respect to tests based on normal variates. The reason that frequencies with respect to tests based on normal variates do not appear in Table 8 is that because of a programming problem, the early simulation runs on which the frequencies with respect to 4, 6, 8, and 10 subjects in Table 8 are based, did not allow for the influence of non-null items effects to show itself in the tests based on the normal variates. The simulation runs for 12 subjects, however, did not suffer from the above limitation and those data will be presented in Table 16. From the analysis of the frequencies in Table 8, it was 63 mm 09 ooH HoH soH moH . so mHH sHH s as Hm sHH HHH onH 00H Ho nsH ~o~ H so a so ooH HHH HHH ooH HHH Ho OHH scH H H.a¢ on oHH HHH HoH ooH os~ on HsH OOH H s s »H 0 HH an oH sH no s HH o sH on s “H Hm a HH as H . H OH an H ”H op «H “H as H uuxra s 3H on H «H ss HH oH as H sH Hs aoH so HsH HHH ss soH HHH s on on HHH Ho HsH HoH as on HoH H . as so nnH so nHH soH on ooH HHH H aszae on HHH oo~ HH oHH mnH .Hs no msH H H H o~ s 0H so H OH mm s OH 0 H “H H “H ss m Hm as H a HH Hs s 9H as m oH Hs H uuxra HH HH HH n HH mm s on as H HH HH noH pm HoH onH Hm on mHH s s~ es 00 on mHH HsH on as H~H H . HH HH .Hm so noH HmH on 0H mHH H songs“ Hm so HHH mm on HHH an no HH H o 0 OH 0 HH oH sH on He s m o o os ~H : mm .1 o~ on m 0 HH on oH mH Hs a HH Hs H uoxaa H oH Hm H OH sH HH on as H sH sm sh Hm no sHH sH so sHH s as on soH sH ms sHH sH Ho mHH H 30.; on Hm Hm mH Hs oHH OH on as H u a HH on so om no OHH Hm so ooH H o o 3 ~ s~ on 1 on 3 s c o H HH oH oH mm oH sm Hm H o 0 pm s 0H on a HH so H uuxHa a HH Hs H oH on m mH Hm H 0 0H 0H a HH so m OH HO s H Hm sm 0 0H so sH HH sH H so.a o n sH sH Hs HH «H on on H v H 0 HH ss HH Ho mHH sH mm HH H m m s H 0 HH m HH ms s s H . H HH H. o sH s HH om n s m OH H s HH s QH es H waxHa H «H Hs HH HH Hs s 0H Hm H , .uuz .mmm mausH ”was” no #364 un. .0: OH HH an oH HH on OH «H on oooH. xv H. H. H. «so . Ho.HuHHHp-poua i 23:: 50¢ .9083 «page: vaucoaom vac scuuucuuucu .3532 use: ...an ado-Heuosuwo 05 co pecan .aounowox condoned .HOu m on» we .3933“ .3300?“ 8075 5 {Scams—Ho: Haogunam 05. .m wanna 64 concluded that there were significant main effects due to: (l) the probability of a one, (2) the number of subjects, and (3) items fixed and null in effect versus items random and nOn-null in effect. There were also significant inter- actions between the probability of a one and items fixed vs. random and a significant interaction between the number of subjects and items fixed vs. random. Both of these inter— actions were represented on one graph, Figure 2. The horizontal lines in Figure 2 are .95 probability limits about 50 for means such as those graphed. An inspection of Figure 2 indicates that a favorable comparison of nominal a and empirical probabilities of a Type I error occurred when items were fixed and null in effect and the probability of a one was .5. Also a favorable comparison occurred when the items were fixed and null, the probability of'a one was .2 and there were 10 or more subjects. In the absence of the above conditions, however, only unfavorable comparisons resulted. The overall vector of mean frequencies for Table 8 was 86.6, 48.5, 24.0. The marginal mean vectors for items fixed-null vs. random-~non-null were 40.0, 18.0, 6.3 and 133.2, 78.0, 41.7 respectively. The large mean frequencies for the items random non-null conditions confirm the con- tention made in Chapter I that non-null items effects associated with a design in which items are nested within repeated measures will cause the regular variance ratio test for repeated measures to have too many Type I errors. 240 200 2 I60 0 ‘8 m m c o ’3 |20 o w '8 m 8° .5 .3 4O 8 m s U' m H CH 0 Figure 2. 65 Number of Subjects |2=0 6:0 |Q=o 4:* 8==U Random Fixed l 1 1 0.5 0.2 O.| Probability of a One The interaction of the probability of a one, the number of subjects, and items fixed vs. random, with respect to the data in Table 8. 66 Note that when items are fixed-null the results in Table 8 are generally the same as those in Table 7, but when the item effect is random non-null the situation is quite different. The marginal mean frequencies in d = .05 rejection regions for the probability of a one equal to .5, .2, and .1 were 91.9, 98.7, and 69.1 respectively, and for the number of subjects equal to 4, 6, 8, 10, and 12 they were 44.6, 74.0, 82.6, 104.3, and 127.5 respectively. Tables 9 and 10 contain the frequencies in a = .05, .025, and .01 rejection regions of the variance ratio test statistics for tests of repeated measures effects when the data were simulated under the non-null repeated measures effect conditions indicated in Chapter IV. Table 9 contains, frequencies with respect to tests based on both normal and dichotomous data, whereas the frequencies with respect to normal data were omitted from Table 10 for the same reason that frequencies with respect to normal data were omitted from Table 8. The data in Table 9 which were with respect to normal variates with overall mean vector 517.5, 418.6, 308.7 were significantly different from the data in Table 9 which were with respect to dichotomous variates and which had an overall mean vector of 262.3, 185.6, 114.2. The data in Table 9 with respect to normal and dichotomous data were subjected to separate multivariate analyses of variance and very similar significant effects were found. For both 67 soH es HHs Ho Hos oeH eHe HeH HHH new HHe Hes Hso HHs coo Hen Hoo eoe s OHH Hs es“ oHH Hes HHH ems eHH Hse noH HHH ens nee HHH HHH HHe eeo eee H HHH ee sou nHH ooH eeH Hos eeH eee HeH see eHH eeo osH ”He Hos HHo nee H aosza HHH s HHH HHH oeH oeH osH HHH Hes neH Hee HeH oHH HHH HHH HHs HHo HHe H How so mos moH oos eoH sHe emH so» o~H sew sec eHo ess oeo eHe Heo He» s HH HVH eH HHH AH ses HsH HHH HoH eee HeH HHH eoH Hoe ooH oHo HHe HHo Hee H ooH ss HoH so ooH eHH Hos HHH eee ceH Hoe HsH eso eeH Hoe ooH eHo HHe H eexrw HsH es ssH en HsH eeH osH sHH Hoe HeH ooe eeH ooe HHH eHe sos HoH HHe H HeH oH sos He Hes ooH HHH HHH ooH HHH eHe esH eon eoH Hso HHH eoo HHe s eHH es een eH Hss -H Hss ~nH Hee Hem oHo Hem Ho» oeH ooe HHe eso use H H}. as HHH eo HHs eHH 2H ooH HHH esH Hse HHH Hoe HHH HHH 9.5 see see H .383. oaH en HHH an HeH HeH sHH HeH Hss HsH oHe oeH one HoH eso HHs OHH eHe H 3H» 0H Hos HH oos eHH oHe HeH ooH eeH one sHs sne HHH sHo HHe HHo Hee s oH Haw 3H HHH es enH eoH ses HHH Hoe HHH eHo ooH one HHH eHe Hes HHo eee H 33H sq ooH so eoH HsH eoH eoH ooe HeH Hoe eHH oee HoH ooH eHs OHH ese H vexaw osH nu oHH HH HeH HHH ssH HeH Hes .esH Hee HHH HHe HeH eHH Hos oHH eHH H oeH HH HoH on oHs He eHH HHH eHe oHH .oee HsH nee HHH HHe oeH Hoe oes s HHH e oHH DH osH no eOH eo eHs HeH oee oHH HHH HHH eHH HsH eHe Hes H HHH 3 H3 2. HuH Hs HeH no HHH CH Hoe esH oHe HHH ooe oeH eeo Hes H .535 H» HH HoH HH esH He HoH eoH oHH oHH oHs eeH sss HHH son HHH HHH sHs H HaH H ooH HH seH so HHH oo e~e eoH oee HHH soo HHH HHH HHH ooH oHe s H HnH HH Hsu Hs eHH eh HOH se eHs oeH HsH eeH ooH eOH HHH osH oHH Hes H 0» HH eHH Hs soH He HHH eHH HHH eeH Hes seH HHH OOH oHe oHH ooH Hes H suave ooH eH sHH Hs sew He HOH oo HHH HHH oHs HeH oHs eHH Hee eoH eoe HHs H Ho 0 HHH HH HHH He HHH He HeH oHH eHe HOH Hes esH Hoe HHH HHH esH s H0 “H HnH eH eHH oe oHH He HHH oo HoH seH osH HHH HsH esH eHe osH H Ho s eHH HH HHH Hs HHH Hs eHH Ho oeH HeH oHH Ho HHs HOH Hee ooH H aezaa o: oH HeH oH seH se eoH Hs ooH om HeH HHH HHH Ho HeH ooH eHe HoH H as H eeH eH HHH oe ooH Hs HeH soH Hoe HHH eHs eHH sHe eeH HHH HHH s e HH H HsH o HHH He esH HH eeH oo Hos HsH HeH eHH HHe HHH Hee esH H H: HH oHH sH eoH He eHH eH 0H He HHH HsH HeH sHH eHs HHH one eHH H HfixHo es oH HaH nH HHH oH ooH Hs HoH no HHH HHH HeH HHH HHH HHH HHH OHH H es e eH n esH on on oH 03H HH ooH soH eoH Hs HHH eo ooe oeH s HH HH Ho eH oHH oH HH oH HHH se HsH sHH HHH oe HHH HHH HHH HHH H HH e He HH eHH HH ee oH HHH se oOH soH HsH oH HHH HHH HeH HHH H sass; ss n He HH eHH oH oe HH sHH He HoH OHH eoH ss eoH oo OHH HeH Hs H HH HH HsH HH sH o ooH HH ooH 0H ooH He eHH HHH eHe so~ s s HH e H» OH ooH sH oe eH esH Hs eeH eoH HeH es HHH HoH HHs soH H eH H en o sHH HH Ho HH sHH He HHH HHH oHH He HHH eoH HsH HoH H Hexao sH H HH HH HHH He eH e~ eo Hs ooH oo soH oe OHH HHH oHH ooH H do: .96 lag! HO H0>UA NO .02 z a z a z a z a z a z a z a z a z a Haauoz so uaosoeoeeHa 2 HH on S ...H on 2 on 82 . so -. H. .m. one a so HeHHHeoeomu oDQBOuozuwa can Haauoz .HH92 couuuauoucu .Haaaucoz ouuuuum.ou=udo: vouaonux .vouuouo naouu .uuwa co voucn.uousndaz voucoaom now u ozu mo .uconux acuuuowux oooa.yv a“ .couocoavoum Hauauwaau 0:9 .& vanes ($8 - J eHH HHH sHH HHH mos HHH H:n Hes OoH O oHH mum HHH Ham mu: eHm xnn as; 1HH m floccmm HHH sHH HHH HHH so. Hos ups Hos ems H ooH HOH HHH HoH sHs Hos “so 0 e m e M HH sm HOH eHH eHH HHH Hes es: eHe Hap eH mr rsH HoH oe eoH roH HHe .ee H emxHo ss so HHH HHH OeH HeH eeH OOe HHe H ms mm HHH sHH HeH .eH HHH sos HH H HH osH HHH HsH HHH oes HHs Hee one s we smH esH HmH HeH H’s HoH sse Hee H soccem sHH HOH oeH HHH Hsm eHs eeH Hes eoe H HHH HHH seH oeH HeH Oss HHH see HHH H OH OH H» eHH HHH OH sHs HHH .HH Hee s O es OOH HHH HeH OHH HHH Hes eHm H ewam sH or HsH HeH esH HHH HoH eHs esm H mH HH HHH eoH HeH eHH HeH Hos eHe H HH Ho HsH HHH HoH OHs HHH HHs eke s HH OHH osH ooH HHH oHH ooH ooH HHH H Eoncam os oH HHH HHH oHH ooH “HH HHs HHH H Hs He HsH osH HsH HsH oHH Ops Hes H H o HH 4H oo HoH HHH HHH HHH HHH s HH Hs eH se HeH OHH eHH usH nes H euxHo HH Hs en eHH HHH sHH oHH OHH Hes H eH Hs Hm oo HHH HeH enH eoH OH. H OH He HHH eos ooH meH HHH sHH HHs s Hs oe HHH HH HHH OsH HOH os 1,, es He oHH Ho eHH HmH HHH HHH HHH M saucem os He 3.. H2 H: HrH mHH in 1..H H H HH om Hs sH HHH oHH eeH HHH s e H o He HH eo HsH eHH HHH esH H HH sH Hm eH nu HsH :HH HHH eHH H emme o. HH oH Hs Ho HPH rHH HHH UHH H HH HH on He HH. He eeH HHH s HH oH He HH He re eHH HHH H sH Hs H» HH nHH em HHH HHH H Eoucmm HH oH He He rHH 4H HsH oHH H o “H ,H H HH HO HHH sOH s s m OH 1H m" H: m: HHH seH m H o HH pH He He eOH FHH H nome H HH Ho HH Hs om HHH ooH H .002 . 95m mEflUH m .DDW o Hm>vq we .02 OH HH OH OH HH OH OH HH Om OHOH . ya N. N. h. UGO M NO >HHHHLMDOMA .Hasz coHuumumucH uwmmm _mwunmmwz pwumommm new m wnu _HHSCIGOZ muoouwm whammwz poumwaom .pwummz muwuH Ho .mcoHowm :oHuomem OOOH .mumo msosouoroHn wnu :o . Ho 5. swwHocwsuoum HmoHuHQEm wFH. . o. wanna 69 dichotomous and normal there were significant main effects due to the probability of a one (which you may recall is confounded with degree of non—null effects for repeated measures), the number of subjects, and the level of subject heterogeneity. Also in both there were significant inter- actions between the probability of a one and the number of subjects as well as between the probability of a one and the level of subject heterogeneity. In addition within the data with respect to the normal variates, there was a sig— nificant first order interaction between the number of subjects and the level of subject heterogeneity, and a second order interaction of the probability of a one, the number of subjects, and the level of subject heterogeneity. To find very similar effects in the data with respect to both normal and dichotomous data is somewhat reassuring, but the relationship of the empirical power of a test based on normal variates to the power of that same test based on the subsequently dichotomized variates requires further analysis. Each frequency in Table 9 which was based on dichoto- mous variates was divided by the corresponding frequency in Table 9 which was based on normal variates to form a new variable which can be considered as the relative power of a test based on subsequently dichotomized variates with respect to the power of the same test based on normal vari- ates before they were dichotomized. The relative power data were subjected to a multivariate analysis of variance and 70 three significant main effects and no interactions were found. The significant effects were (1) probability of a one, (2) the number of subjects, and (3) the level of sub- ject heterogeneity. The overall mean vector of relative power variables was .45, .38, .32, which indicates that the relative power of the variance ratio test for repeated measures effects decreases as the nominal a level decreases, from .05, to .025, to .01. There was no interaction between nominal a level and the above significant effects so marginal mean relative power will be reported for d = .05 only. For probabilities of a one equal to .5, .2, and .1 the mean relative powers for a = .05 were .58, .49, and .28 respec- tively. For numbers of subjects 4, 6, 8, 10, and 12, the means were .39, .41, .43, .50, and .54. For the four levels of subject heterogeneity l, 2, 3, and 4 the means were .52, .46, .43, and .41 respectively. The differences in mean relative power show clear trends. As the probability of a one becomes smaller so does the relative power, which is the opposite of the trend which might have been expected, since as you may recall from Chapter IV the degree of non-null effect in the simulated data was selected to counter the effect of decreased vari- ance corresponding to a decreased probability of a one. Thus the power of tests based on the dichotomous variates should not have changed across levels of a probability of a one, whereas the power of the test based on normals should have and did decrease across levels of a probability of a 71 one (and decreasing non—null effects). The results indicate however, that the power of tests based on the dichotomous variates fell off more rapidly across levels of the proba- bility of a one than did the power of tests based on the normals. The explanation of the above may be that as the probability of a one becomes smaller the point of dichoto- mization is such that more of the "information" carried in the normals is lost.6 Another clear trend is that as the number of subjects increases, the relative power does so as well. The trend with respect to the number of subjects is most likely a function of the effect of the central limit theorem. The third trend indicates a loss in relative power with an increase in subject heterogeneity, a trend for which this investigator has at present no explanation. A multivariate analysis of variance of the frequencies in Table 10 disclosed significant differences for almost all sources of variation. The interaction of items fixed vs. random and the level of subject heterogeneity was not sig- nificant nor were the two second order interactions which included the above two sources, the probability of a one by items fixed vs. random by subject heterogeneity and number of subjects by items fixed vs. random by subject hetero- geneity, but tests of all other sources indicated significant differences. Relative power ratio variables were formed for 6Recall from Chapter IV that the non-null repeated measure effects were added in before rather than after dichotomization. 72 all of the situations with respect to Table 10 for which there were frequencies based on normal variates that were not inappropriate.7 The relative power ratio variables that could be formed were analyzed by means of a multivariate analysis of variance with empty cells. The deficiency of the above analysis compared to the like analysis performed for Table 8 can be expressed in terms of which sources of variation are untestable due to empty cells. The untestable sources of variation are: all of the second order inter- actions and the first order interaction of number of sub- jects and items fixed vs. random. Of the testable sources of variation significant differences were found for: (1) the probability of a one, (2) the number of subjects, (3) items fixed vs. random, and (4) the level of subject heterogeneity. The overall mean vector of relative power was .47, .41, .34. The marginal mean relative powers for a = .05 regions for probabilities of a one .5, .2, and .1 were .62, .51, and .30 respectively; for numbers of subjects 4, 6, 8, 10, and 12 they were .37, .41, .44, .50, and .54 respectively; for items fixed .45, for items random, .59; and for levels of subject heterogeneity 1, 2, 3, and 4 they were .53, .48, .45, and .44 respectively. The same trends in relative power were found in this analysis as were found in the above mentioned analysis of relative power plus an 7 . . . Inappropriate frequenc1es based on normal variates occurred where the number of subjects was fewer than 12 and items were random-~non—null. 73 effect for items fixed vs. random. The evidence that the relative power is greater for items random than for items fixed under Design 3 is unimportant, however, since other evidence strongly indicates the variance ratio test for repeated measures is too liberal under Design 3 when the null hypothesis is true and items are random. Recall at this point that in Chapter I it was indica- ted that Satterthwaite's "synthetic variance ratios" or "quasi"-F tests could provide an appropriate test for repeated measures effects when in Design 2 there was a non- null items by repeated measures interaction or when in Design 3 there was a non-null items effect. The most startling result of this study concerned the empirical testing of the above contention with respect to dichotomous data. Tables 11 and 12 contain the frequencies in a = .05, .025, and .01 rejection regions of the quasi-F ratio test statistics for tests of repeated measures effects when the data were simulated under null repeated measures effect conditions and null subject by repeated measure interaction effect conditions. That is, Tables 11 and 12 are the quasi-F analogs to Tables 7 and 8. The data in Table 11 which were with respect to normal variates with overall mean vector 36.5, 16.6, 5.6, were significantly different from the data in Table 11 which were with respect to dichotomous variates with overall mean vector 65.9, 38.1, 18.2. The data in Table 11 with respect 74 s HH HH OH sH He H Os HH HH sH eo H HH OH Hs HH HH s H HH sH ...H OH Ho H OH HH Hs HH HO s HH HH HH Hs OH H H OH OH Hs HH HOH H oH OH os HH He H sH OH Hs HH sH H sesas e Os HH sH HH HHH s OH sH HH HH ee H sH OH OH OH Ho H H HH oH HH HH sH O HH H OH OH OH O HH e oH OH OH s HH e HH sH HH HH Ho H HH OH ss HH Ho H HH o OH OH so H H HH HH HO HH oe H OH OH HH sH sH H HH OH OH es HH H HHHHH H Hs eH sH OH HO H HH OH OH HH Oo H HH HH He HH HOH H H HH HH Hs Hs HO H HH o eH sH HHH o HH HH OH HH HO s s OH O HH on ~HO ~ on N no Hm mo e s~ NH .3 ON sm H H OH O Hs sH HH s HH H HH eH HH H HH HH HH HH OH H aoszs H OH OH HH HH He H H s OH oH OH H OH HH He sH Ho H s e HH HH oH HH H HH H Hs sH HH s HH sH es eH O s OH H HH O OH HH HH o oH HH HH Hs sOH H HH HH HH HH Ho H H HH O HO OH OHH H HH H HH eH so H HH O He HH Ho H HOHOH H H H sH sH Hs e HH e eH oH so OH HH oH oH OH Ho H O O H OH HH HH O oH o OH OH oe O eH H os sH se s O HH HH OH sH es O o H sH HH OH H HH O Hs HH HH H O HH HH OH HH eo O OH H OH oH sH OH OH HH Hs eH HH H aoEaH O Hs o HH HH HO O oH H sH HH HH H OH OH OH HH He H O e HH OH HH HH O H OH HH HH OH H HH HH Hs HH He s H O HH H sH HH OH O sH HH es HH HH H sH oH ss HH HH H O o sH HH eH Hs O HH o oH eH HH H OH HH HH OH HO H OOHHH O HH O OH sH Os O HH HH Os HH HH H oH HH HH sH HOH H O H OH eH OH sH H HH oH OH Hs HH e HH HH Hs OH re s O O HH HH HH OH H HH OH ss os Oe e eH HH os Hs HH H H s eH HH Hs OH O eH HH HH es Os H OH HH HH ss Oe H 30232 H O HH H Hs Hs H sH oH OH Hs es H HH OH Hs OH HH H s O HH H Os HH H sH OH HH Os os H oH OH Hs eH HH s e s H OH O HH eH H eH oH H. HH HH O OH HH es Hs HH H H H HH OH OH Hs H OH HH OH HH OH H sH HH Hs sH HH H OOHHH H HH HH oH HH Hs H HH HH eH HH sH H OH OH Hs oH HH H HH s HH sH OH OH oH H OH HH He HH HH HH HH OH ss HH s HH o HH HH HH HH OH HH HH HH HH HH H HH HH HH HH OH H HH H HH H on H OH OH HH eH Hs sH HH HH HH HH HH HH H geese sH O HH O HH H HH o HH OH HH HO o HH HH Hs HH He H sH O HH H He e OH O HH HH eH HH HH OH HH eH se HH s s n 0 ON n as 0 4H m em 0 so On m NH on on Os mo m sH H OH s sH eH e HH HH OH Hs He HH oH eH HH Hs He H OOHHH HH H HH e Oe OH HH o oH HH oH Hs HH HH HH HH Hs HH H .umm .nsm oaouH m.mmw\ uo HU>0H mo .02 z a 2 O z O 2 O z O 2 O 2 O 2 O z O Hesuoz Ho OsoaeeoeeHO OH HH OH OH HH OH OH HH OH OOOH .OO H. H. H. ego O HO HOHHHOHOOHH .Hasz zoom mucouuu nousuau: kumonom vac coauoououcu .vommouu HEMUH .mHmo anoEOHOLUHQ vac Haahoz ao venom .oonaudo: vouqonum How maHuasc onu we .ucoamox coHuuonux ooo~.:§ :H .moHucusauum HmuHHHaam och .HH qumH 75 HH HO HOH HH oe HHH OH OHH OHH O .3 He HOH HH HH HO OH OH OHH H 83.3. OH OOH HHH OH HO HH He HO HHH H HO HO HOH OH HO HO OH HH HHH H H 3 HH HH HH O: HO HH oe HO O H OH OH HOH OH HO HO HH OO HH H OOH: HH OH eO OH HH HO OH HO OH H HH O: OO OH OH HOH OH HO HOH H OH OH HO HO HH HHH OH eO HO O OH HO HO OH HO OHH OH OH HHH H OH OH OH HH HO OHH HO HO HOH H aaOOOH HH HH OH HH He eO HH OH eOH H HH OH He HH OH HH HH HH OO O OH HH OH ee HH OH OH HH OH HO H OOHH HO OH HO OH OH OH HH HH OOH H H HH HH HH HH HH OO HH OH HO H OH OH HO HH eO eO OH HO OHH O HH OH HO OH HO HHH HH HOH OOH H HH HH OO HH OH HO OH HH HO H aoeaa O HH OH HH HO OH HO He HOH H H OH HH OH HO HO OH HO He O O HH OH HH OH HH HO OH OH ee H OOHHH HH OH OH O OH HH OH OH HH H HH HH OH OH OH OH OH HH HO H HH HH HO HH HH Oe OH O; HH H O HH OH HH HO He HH OH OH H O HH. HH HH HH He OH HH 3 H .825. HH OH HO HH HO He OH He OOH H H OH OH HH HH HH OH HO HO O O H HH HH HH HH OH HH OH HO H OH OH OH HO OH OH HO HH OH HO H O HO H OH HH HH eH He HH HO HH H O HH OH O OH HH HH HH OH O HH OH HH H HH HO HH HH «e H ao_. H OH HH HH OH OH OH OH Oe H HHH O O OH eH HH HO OH OH OH H O H e H O OH O HH HO O O H O H e HH HH HH OH HO H O O H HH HH OH HH HH OH H HOHOH O H O O HH HH OH HH He H .Hox .HOO mmme O.HOH mo «H53 we .02 OH HH OH OH HH OH OH OH OH OOOH ..o. H. H. H. 25 O Ho UHHHOOHSO 2:3: 50m 300qu 0.323: vouooamm wad cowuoauuucu 6332 use: .33: 090830539 :0 «53¢ Johan-or @0333. new unwound 05 mo .oaOHwom aoauuonox 003 ..o 5 .aouucoakum Hauauaam 05. .2 Snub 76 to normal and dichotomous data were then put to separate multivariate analyses of variance. The analysis of the data with respect to the dichoto- mous variates indicated significant effects due to: (1) the probability of a one, (2) the number of subjects, the inter- action of (l) and (2), and the interaction of (l) and items fixed vs. random. The marginal mean frequencies in a = .05 regions for probabilities of a one equal to .5, .2, and .l were 77.7, 67.1, and 52.8 respectively and for the number of subjects equal to 4, 6, 8, 10, and 12 the marginal mean frequencies were 38.6, 57.5, 63.9, 83.6,and 85.8 all respectively. The above two significant interactions are represented in Figures 3 and 4 respectively. Both figures indicate that the empirical probability of a Type I error is not in general close to .05. Figure 3 indicates that although the frequency in the rejection region is less affected by the probability of a one as the number of sub- jects increases, the tests become rather liberal. Figure 4 is self eXplanatory. The analysis of the data in Table 11 with respect to the normal variates was interesting in that it tends to contradict earlier findings. The analysis indicated a strong significant effect due to the number of subjects. The marginal mean frequencies in a = .05 regions were 53.8, 43.5, 24.3, 32.2 and 28.8 for 4, 6, 8, 10, and 12 subjects respectively, which indicates a general downward trend in the empirical probability of a Type I error with an increase ICK3r 90- k 80- 2 .9. O 32 70L— C. O .3 0 g 60-— (D n: u 50— .fi SUBJECTS a) 40— 2 l2=o Q) :3 8‘30— 10:0 1... Lu 8==I 20— 6:0 4=* |O- cfi: l l l 0.5 0.2 O.| Probability of a One Figure 3. The interaction of the probability of a one and the number of subjects, with respect to the data in Table 11. 78 90r- m 80- c o -H m m m 8 .H 70- 4..) o m ’f‘fi m m Ln 0 . 60- u c -:--l am) 50*- —-—— Random \ .8 \ c; \ w Fixed 9 U m H O 4o- ‘7 C) l l l 0.5 0.2 O.| Probability of a One Figure 4. The interaction of the probability of a one and items fixed vs. random, with respect to the data in Table 11. 79 in number of subjects. The data in Table 12 were analyzed by means of a multivariate analysis of variance and significant effects were found with respect to: (l) the probability of a one, (2) the number of subjects, (3) items fixed vs. random, and the interaction of (1) and (2). The overall mean vector for Table 11 was 72.8, 42.5, 22.3. For a = .05 regions the marginal mean frequencies for probabilities of a one equal to .5, .2, and .l were 88.4, 72.5, and 57.2 respectively, for numbers of subjects 4, 6, 8, 10, and 12 they were 37.5, 50.7, 75.6, 85.5, and 104.4 respectively, for items fixed and null the marginal mean frequency was 65.0 and for items random non-null it was 80.5. The significant interaction is represented in Figure 5 which is somewhat similar to Figure 3 and which in general lends itself to the same interpretation. Tables 13 and 14 contain frequencies in a = .05, .025, and .01 rejection regions of the quasi-F test statistics for tests of repeated measure effects when the data were simu— lated under non-null repeated measures effect conditions. Thus Tables 13 and 14 are the quasi-F analogs of Tables 8 and 9. The results in Tables 13 and 14 are most startling for it is apparent that although the quasi-F tests based on normal variates responded in an appropriate manner to non— null effects that the quasi—F tests based on dichotomous variates did not. The quasi-F test based on dichotomous variates has significantly fewer frequencies in rejection 80 IIOF l00- C—— 90— 2 80- o a: 5 7o- .3 O .0) ‘a? ‘1‘ 60- ll 50- .5 g 40- 3 8‘ Number of Subjects $4 I“ 30L- I2=o 6:0 IO=O 4=* 20- 8=- 01L L l J 0.5 0.2 0.| Probability of a One Figure 5. The interaction of the probability of a one and the number of subjects, with respect to the data in Table 12. 81 OHH OH HHH OH OHH HH HOH H HOO H OHe O OHH H OHO H HOO H O OHH H OHH O OHH HO OOH O HOH H OOH O HHH O HOO H HOH H H aogzd HH e OOH HH HOH HO HHH O OOH O OOO HH OHH O HHO O OH H H HO HH HHH OH HeH HH HOH O OOH O HOO O OHH O HHH O OHH O H HOH HH HOH HH OHH HO HOH O OHH O eee O OHH O OHO O HHO O O HH HHH OH HHH HH OHH OO OHH H OOO O OOH O HHO O HHH O OOO O H OOOOO HO OH OHH nH OHH OH OOH H HHH O HeO HH HOO O OHe H HOO H H HH OH OHH HH HHH eO HHH H HHH e HHO OH OOO O HHe O. OHH H H . HOH H HHH HH HOH HH HHH O HOO H HHO HH OOO O OOH H HHO H O HOH O HHH OH OHH OH HHH O HHH HH eHH HH HHH O OHH O HHO H H Exaqx OHH HH OOH OH OOH HO OHH H OHH OH OOO OH OOO O OHO H OOH H H HO HH OOH OH HHH O HOH H HOH OH OHH OH HOH H OH H OOO H H HOH H HHH H HOH OH OOH O HHO H OHO HH OHO O O)H O OH H O OH HOH O HOH H OOH H OOH H HHH e HHH HH HHH H HOH H HHO H H OssH HOH OH OOH HH OOH OH OHH e HOH OH HOO OH eHO O HOH O HHH H H HH OH HOH OH OHH eH HOH H HOH H OHH O HHH O OHH e OOO O H OOH O HHH OH OOH HH HHH O OOH OH OHH eH HHH O HHO H OOH O O HO OH HHH OH HOH HH OOH O OOH HH OHO OH OOH H OHH O HHO HH H aoeaa OH OH HHH OH HHH HH HOH H HOH O HHH OH OHH H HOO H OHe HH H OH OH HHH OH HOH HO OHH OH OHH HH HHH OH HHH H HOO O HOH H H HHH OH eHH HH HOH HH OHH H OHH OH HHH HH OHH H HOO H HOH H O O HO H OHH H HHH OH HHH O OHH O OHO OH HOH O HHH H HOO O H OSJH On HH OOH HH OOH HO HOH e HOH HH HOH HH OOH O HOO H OHe H H OO H HHH OH OHH OO HHH O OHH OH OOH HH HHH O OOH O OOH OH H O3 OH OHH HH HHH OH HOH HH OOH OH HOO HH OHH H HHO H HOO H O OO OH HO OH OHH OH OO OH HOH OH HOH OH OHH H HHO O OOH H H aoEBO HO O HO OH HOH OO OO H OOH O HHH OH HOH H OOH O OHO O H OH OH OO OH HHH HO OH O HHH HH OHH HH HOH O OOH H HOO O H OO H OHH O OHH OH OOH H OeH e eOH HH OHH H OHH H OHe e O O HO O HO OH HHH HH HHH H HOH H OOH OH HHH H OOO e HOH HH H Ooxrw OO O OH H OeH HO HO H HOH O OOH HH HHH H HOH H OHO H H OH OH HO HH HOH HO HO O OOH HH HHH HH OHH H HHH O HHO OH H H: H OOH H HHH H OH O OOH H OOH HH OHH H HOH H HOO HH O HH H HO O OHH OH OH H HOH HH OOH HH OHH H HHH O eOH HH H aoEaO HH H He H OHH HH OH H OOH H HOH OH HHH H HHH HH HOH HH H HH H HH O HHH OH OH OH HOH OH HOH HH HHH H HOH OH OOH OH H O: O OO H HHH H HO H OOH H OOH HH OHH H OHH H OOO O O O HH H O0 O OOH H HO O HHH HH OOH OH HHH O OHH H OHH HH H OOOOH OH O ee 0 OOH O HO H OHH eH OOH OH OHH H HHH O OHH OH H OH HH He HH HHH OH HO H HO OH OOH O HOH e OHH HH OOH OH H .HflfllmmWI:lmmaH||:41O:WIII mo H0>04 MO .02 2 O 2 O 2 O 2 O 2 O 2 O a O O O 2 O HOsOoz Ho OneseHOOeHO OH HH OH OH HH OH OH HH OH OOOH.OO H. a. m. 25 O Ho HOSE-noun and Haauoz no voowm .Oouamaoz vouaaaux Haw muwnaso on» no .aonom ceauoonum oooasxv on» cH .OOHucoaqum HOOHHHaam och .Haaz Goduuquoucm .Haaancoz nu00uum museum: umumuaum .vommouu mEouH .muao m5080uonHo .mH uaan 82 OH HO HO O OO OO O O OH O HH HO HO H OH HH O oH OH H OO OH HO HH OH OH H H O H .333. HH OH HH HH OH HO H O O H HH OH HO O O O O o o O HH O HH HH H H O o H H H O OH HO H H O o H H H OoxHO HH HH HH H H O o o H H H H H O OH OH H H O O HH OH HH HH HH HO H H O H HH HH OO HH OH HH H O O H aoeaa HH HH HH OH OH OH H OH OH H HH OH OH o H OH O O o O OH o O OH H HH HH 0 H H H O HH HH H H O H H O H OOHHO OH HH HO H O OH H H H H HH OH OH OH HH OO O HH HH O HH OH HO OH OH HO O OH HH H oH OH OH OH OH OH O HH OH H aoeax HH OH HH HO HO OO H OH OH H OH OH HH O OH HO O H H O O O HH OH H O HH H O O H HH HH OO H O HH O H H H OOHHO H OH OH H O HH H H O H O OH OH HH HH. OH H O OH O O OH OH OH HH OO H O HH H O O HH 3 HH OH H O 2 H 833. HH HH OO HH HH OO H HH HH H H HH OH H O HH H H OH O O O HH OH H OH OH H H HH H O HH OH H O HH H O OH H OOHHO O HO HO O HH OH O O HH H H H HH O HH HH H O OH O H O HH O O OH H HH OH H H O HH HH OH HO O HH OH H 322.3. H H OH HH HH OH O HH OH H o H H o O O O H O O O o H HH H HH OH H O O H o n n ~ nu m~ N O NH N wome O OH HH O O HH O OH HH H .OOO .OOO InmmmmH O.OOO uo Hw>uq mo .02 OH HH OH OH HH OH OH HH OH OOOH..O. H. H. H. mum‘s Ho OOHHHOOOOHO .33. 3330503 no wanna .cgaudut v3.33— uOu HHH-2.6 05 no . 2:52 nouuoIHUunH ..Saclcoz caucuuu «HHH-cu! vouooaum €3.32 .3qu acumen canon—.08 c8.— .3 5 .oouucoscoum #35395 05. .T. 393. 83 regions when the effect it is testing is non-null than when that effect is null and in the cases investigated the empirical power of the quasi-F test is consistently less than the nominal a level! The data in Table 13 with respect to normal variates has an overall mean vector of 406.6, 300.2, 195.4 which is significantly less than the mean vector of the data with respect to normal variates in Table 9. The relative power of the quasi-F based on normal variates to the regular F was not analyzed, but it can be seen that it is somewhwere between .8 and .6 depending on conditions. The frequencies in Table 13 based on normal variates were analyzed in the same manner as in Table 9 and the same significant effects were found. Relative power ratio variables were formed with respect to the data in Table 13 based on dichotomous fre- quencies and an overall mean vector was calculated. It was .10, .08, .06. Further analysis of the data in Table 13 or analysis of the data in Table 14 appeared superfluous and was not done. Table 15 is a rearrangement of data which has been previously presented. The arrangement of data in Table 15 was established to allow easy contrast of data with respect to regular F and quasi-F tests based on normal and dichoto- mous variates under null and non-null repeated measures conditions. Table 16 is laid out in the same manner as Table 15 84 OOH O HHH HO OHH HH HOH H HOO H OHO O OOH H OHO H HOO H O OHH H OHH O OHH HH OOH O HOH H OOH O HHH O HOO H. HOH H H aozz HO O OOH HH HOH HO HHH O OOH O OOO HH OHH O OHO O HHH H H H O H0 H HHH OH HOH HH HOH O OOH O HOO O HHH O HOH O OHH O H O OH HH HOH HH OHH HO HOH O OHH O OOO O OHH O OHO O HHO O O -H2:é HHH OH HHH HH OHH OO OHH H OOO O OOH O HHO O HHH O OOO O H H6 HH OHH HH OHH OH OOH H HHH O HOO HH HOO O OHO H HOO H H OesO HH OH OHH HH HHH OO HHH H HHH O HHO OH OOO O HHO O OHH H H HmpHcou Icoz OHH HO HHO HO HOO OOH HHO HOH HHH OOH OHO HOO HOO OHO OHO HOH HOO OOO O OHH HO OOH OHH HOO HHH OOO OHH HOO OOH HHH HOO OHO HOH OHO OHH OOO OOO H HHH HO OOH HHH OOH OOH OOO HOH OHH HOH OOO OHH OOH OOH OHO HOO HHO HOH H a02:£ HHH OH HHH HHH OHH OHH OOH HHH HOO HHH .OHH HOH OHH HHH HHO HHO HH HHO H HOH OH HOO HOH OOO OHH OHO OOH OHH OHH OHO OHO OHO OOO OOO OHO OOO HHH O O HHH OH HHH HH OOO HOH HHH HOH OHO OOH HHH OOH HHO HOH OHO HH HHO HOO H OHH OO HHH OO OHH HHH HOO HHH OHH OHH HHO HOH OOH OHH OHO OOH OHO HHO H OOHHO HOH H: OOH HO HOH OHH OOH OHH HOH HOH OOO HOH OOO HHH OHO OOO OOO HHO H O OH HH OH OH OO H OO HH HH OH OO H HH OH HO HH HH O H HH OH OH OH HO H OH HH HO H HO O HH HH HH HO OH H H OH OH OO HH HOH H OH OH OO HH HO H OH OH HO HH OH H OOBSO O OO HH OH HO HHH O OH OH HH HH OO H OH OH OH OH OO H H HH OH HH HH OH O HH H OH OH OH O HH O OH OH OO O O O HH OH HH HH HO H HH OH OO OH HO H HH O OH OH OO H .;HO:O H HH OH HO HH OO H OH OH HH OH OH H HH OH OH OO HO H OOO: H HO OH OH OH HO H HH OH OH HH OO H HH HH HO HH HOH H chucwu O O OH OH OH OO H _ OH OH HH OO OO O OH OO HH HO OO O H HH HH HH HH HO OH HH OH HH OH OO OH HH HO OH OO HH H H OH OH OH HO OH OH HH OH OH HO HH O O HH OH OH OO H OOEEO O OH OH HO HO HH OH HH HH OH OH O OH H HH OH HH OH H O O OH O OH HH H O OH HH OO OH HH OH H OH HH HO O O HH O OH OH HO OH HH O OH H OH HH O O H H HH HO H O H HH OH HO OH O H OH OH OO OH OH HH OH H HO HO H OOxHO O O OH OH HH OO O H HH HH HO OO HH HH H H HH OO H no: .nam HEOHH u mwusweu: no Ho>0H vouaoawm O O O O 2 O 2 O O O x O 2 O 2 O O O .OOEOOOOOHO HO Hastoz OH HH OH 2 HH OH OH H H OH 82 . x. a. N. n. «:0 O Ho OOHHHOHOOOO van Haapoz co OOOom .Oouaanz vuuaoaqm you OIHOoao can u ozu no .nconom coHuounum oooH.yU :H .OoHucozcuHm HauHuHaam vfia .ouuonnsm 0>H039 .vomOoHu anuH .cuao OnosoHOLOHc wuH OHOOH 85 HHH OH HOH HO OHH OO HOH O OHH OO HHH OO OHO O HHH O HHO OH O HHH HH HHH HH OOH HO OOH H HHH O HHH HH HOH O OHH OH HOH OH H 90 3: OHH OH 0: Ho H3 2 OH: OH OHH .Om OHH H OOO H HHH m H 5252 O OHH HH OHH OH HOH HH OHH HH HHH OH HOO HO OHH H OOO O OHH O H -HOEH. HOH HH OOH OH OOH HO OOO O HOH O OOO O HOO O OHO O OOO O O HHH O OHH HH HHH HH HOH H HHH H OHO O OOH O OHO H OOO H H HO O HOH OH HOH HO HHH H HHO H HHH O OOO O HOH H OOO H H OBHO OO HH OOH HH OHH HH OHH H OOH H OOO O HOH O OOO O HOH H H - IIII u n H=chuv OHO OHH OOH HHH OHO OOH HOH HHH HHO HOO HOO HHH OHO HOH HOO HOO HHO OHH O .Hsz OHH OHH OOO OHH OHH HHH HOH HHH HOH HHO OHO OHH OOO OOH HOO OOO OHO OHH H OH: HOH HOH OHH HOH HHH OO HHH OHH OOO OHO OOO HOH OHO HOO OOH OOO OOO H 5O25O OOO OOH OOH HOH OHO HHH HOH HOH HOO OOO OOOH HOO OOH HOO HHO OHH HOO HHO H HOH OH HOO HOH OOO OHH OHO OOH OHH OHH OHO HHO OHO OOO OOO OHO OOO HHH O O HHH OH HHH HH OOO HOH HHH HOH OHO OOH HHH OOH HHO HOH OHO HHO HHO HOO H OHH OO HOH OO OHH HHH HOO HHH OHH OHH HHO HOH OOH OHH OHO OOH OHO HHO H OesO HOH HO OOH HO HOH OHH OOH OHH HOH HOH OOO HOH OOO HHH OHO OOO OOO HHO H OH HH HOH HO OOH HOH OO HO HO HO OOH HHH OH OH H OHH HO OHH O OO OO HOH OO HHH HOH HO HH HH HH OOH HO OH OH OO OO OO OHH H OO OH o9H HOH OHH HHH HO OH HO HO HHH HH HH OO HH HO OH HHH H egzax OH HO HH HO HO HOH HH OH OO HO OOH HO OH OO OO HH OO HHH H OH OH OH HO OO HH H HH O OO HH HO H HH HH OO OO HO O O O OH OH OH OO HOH O OH HH HO OH HO H HH H OO OH HH H -tsOO HH HH HH OH HO OH O OH OH HH OH HO HH OH HH HO OH OH H OO¥: O HO oH OO OO OO O . OH OH OH OH HoH O OH OH HO oO HoH H HmOH:vU HOO HH OHH OO HHO OOH HHO HOH HOO OOH HHH OOH OOH OO OOH OHH HOH OHH O OOH OO OHO HO OOH OHH HHH HHH OHO OHH OHH OOH HOH HO HOH OOH OHH HOH H HHH OH OHO OO HHH HHH HHH HHH OHO HOH OOO HHH OHH HO HHH OHH HOH OOH H eO2:O HOH OH OOH OHH OOO HHH OOH HOH OHH OOH HOO OHH HHH O HOH HOH OHH OOH H O O OH O OH HH H O OH HH OO OH HH O HH OH HH HO O O HH O OH OH HO OH HH O OH HH OH HH O O HH HH HH HO H O H OH OH HO OH O H OH OH OO OH OH HH OH HH HO HO H HOE: O O OH OH HH OO O H HH HH HO OO HH H H H HH OO H Ono: . 33m MFCuH ..w fizhfitfivpz mo ~m>m4 vouqzzum 2 O 2 O z a 2 O 2 O 2 O 2 O 2 O 2 O .yOOOEOOcOOHO:Hm.HmaOam OH HH OH OH HH OH HH OH OOOH.x. 11 H. H. H. - «:0 O umINHHHHOOOOwHII aasuoz oz» so wanna .omusmmoz vuueoaox no“ unumaao vac m ecu uo .Ocoumox :oHuuuaum oooH.KO :« .aouocoscoum HHOHHHaEm och .ouuonnam 0> 039 . mummz newuw .aumo moosouonuHa vca H v .o. OHOOH 86 and includes some new data, that with respect to normal variates under Design 3. Tables 17 and 18 present frequencies in a = .05, .025, and .01 rejection regions for variance ratio tests of sub- ject by repeated measure interaction effects, when the data were simulated under null repeated measure and null subject by repeated measure interaction effect conditions. The frequency data in Table 17 are with respect to Design 2 and the frequency data in Table 18 with respect to Design 3. A multivariate analysis of variance indicated a sig- nificant difference between the data in Table 17 based on normal variates with overall mean vector 48.4, 24.9, 12.2 and the data in Table 17 based on dichotomous data with overall mean vector 73.3, 45.4, 25.5. Subsequent analysis of the data in Table 17 based on normal variates indicated a significant effect due to the number of subjects and a significant interaction of the number of subjects and the probability of a one, results that were unexpected. A series of post hoc comparisons indicated that the unexpec- ted results occurred only when the data were simulted for 8 subjects and the probability of a one for the dichotomous variates was .1. Since the probability of a one for the dichotomous variates cannot affect the data in Table 17 based on normal variates (this was checked very carefully) it must be concluded that the significant effects found in the frequencies based on normal variates are Type I errors and if the suspect simulations were rerun with a different 87 HH HO OH HO HO OOH OH HO OH HO HO HHH HH OH OH OH HO HO O OH HO OH HH OO HO OH O HO HO HO HO OH HH OH OH OO OO O OH OH OH OH HH OOH O OH OH HO OO HO OH HH O HH OH HO H aoE:a HH HH OH OH OH OO OH HH OH OH HH OH HH OH OH HO HO OH H OH HH OH OHH OH HHH O OH HO OH OH OOH H HH HH HO OO OO O HH HH HH HH HH HH OH HH OH OH HO OO OO HH OH OH OH OH OH H OH HH HO HO OH HHH OH OH HO HH HH OHH OH HH OH HH HH OO H OOOOH HH OH OH HO HO OHH HH OH OH HO HO OO O O OH OH HO HO H HH HH HH OH OH OHH OH HO OH OO OO OO O HH OH HH HO OH O HH HH HH OO HH OHH HH OH OH OH HH OO HH OH OH HH OO OO H OH OH OH HO HO HH OH OH OH OH OH OH O O HH HH OO HO H geese OH OH OH HO OH HHH OH OH HH OH HO HO HH O HH OH HO HO H OH HO HH OHH HH OHH OH HO OH HO HO HOH OH OH OH HO HO HO O OH O HO HH OOH HO HOH HH OH OH HO HO HHH HH HH HH HO HO HO O O HH HH HO HO HHH OH HH HH HO OO HO OH OH HH HH HO OH H OOOOH HH HH OH OH HH HO HH HH OH HO OO OO HH HH OH HO OH OO H O HH OH OO OH OH O HH OH OH _OH OO HH HH HH HH OO OO O H OH OH HO OH OH HH OH OH HO HO HO HH OH OH HH HH OH. H .. OH OH OH OH OH HH OH OH HO OH OH OH OH OH HH HO OO H 8.23 OH OH HH OO HH HO O HH HH OO OH OO HH HH HH OH OH HH H H OO HH HO HO HHH OH OH OH HO OH OO OH OH OH OH HO HH O O H HH OH OH OH OHH HH HH HH HH H HO HH H HH OH HO OO H H HO HH OH HO HHH H OH OH HO OO HO HO OH HH HH OH OH H OoxOH O OH OH HH HH OH H HH OH OH OH HH H O OH OH HO HH H OH HO HH HO OH HH OH HH HO HO HH HO H HH H OO OH HO O OO HO OH HO OH HO HH OH HO HO HH HH O HH OH HO HO HO H HO OH HH HH OO HO HH OH OH HH OH OO O OH OH HH HO OH H aces: OO HH OH HO HO HH O HH OH OH OO HO OH OH H OH HH OH H OH HO OH HH OO HHH HH HH HH OH HH HO HH OH OH OH HH OO O O OH HO HH O, HO OHH O HH OH HH HO HO H HH OH HO OH HO H OH HH HH OH HH HOH HH HH HH HH OO HO H OH O OH HO HO H OOOOH OH H OH HH OO OH H HH HH OH HO OO H OH O OH HO OH H OH HO HH HH HH OO HH HH HH OH HH HO OH OH HH OO HH HH O HH HH OH .H HO HO O HH OH H HO OO HH OH OH HH HH HO H HH OH HH OH O: HO O HH HH OH HO OO HH HH OH OH HO OO H aos:e OH OH OH HH OH HH O HH HH OH HO OH HH OH HH HH OO HH H . H HH HH HH H HO OH OH OH OH OH HO O HH O OH HO OH O O O OH OH HO OH OH HH HO OH OH HO HH H HH HH OO OO HH H HH HH OH OH HH OO O OH OH HO HO HO OH H HH OH OO OH H HaxOH OH HH HH HH OO OH OH HH OH OH OH HO OH OH OH HH HO OH J 133.1% uo Ho>oq .PI. we .02 2 O O O 2 O 2 O 2 O 2 O 2 O O O a O Huauoz Ho OOOEOOOOOHO OH OH OH 2 HH OH OH HH OH 82.3 A. N. n. «so O Ho HOHHHOOOOHO .332 53 caucuum .353an voucoaom van coHuuauOBaH .3305 2.03 .35 30603203 EB 13:02 on... no voomm .coHuoauou:H nouaqaax kuuoaom an unconnsm any you m osu uo .odonom aoHuoonum ooo~.5 aH .OwHucosvuuh HMOHHHQHN och .H‘ oHnuH 88 H OH OH OO HH HOH OH OH OH HO OH OH OH OH OH OH HO OO O OH OO HH OOH HO OHH HH H OO O OH OH O HH OH HH OH OO H EOOOH HH HH OH OO HH HOH OH OH HH HO HO OH OH HH OH HH HH OO H OH OO OH HO OO OH OH HH OH HO OH HO OH HH HO HO OH HO H HH OO HH HHH HH HOH OH HO OH OH OH OOH HH OH OH HO HH OH O HH OH HO OH O HO HO OH HH HH HO OH OO HH HH HH HO HH OH H OOOH» OH OH HH OH HO OOH OH OH OH OH OO HOH HH OH OH OH OH OO H HH HO HH HO OO OHH HH OH OH OO HO HO OH OH OH HH OH OH H OH HO OH OHH HO HOH OH OH HO HH HO HO HH OH HH HO OH OO O HH HO HH OH HO OHH HH OH HH HH HO OOH OH O HH OH OO HO H :2EOH OH HO OO HH HH OOH HH HH OH HO HO OH OH HH HH HH OH OH H HH OH HH HO OH HH HH OH HO HO HO OO HH OH OH OH OH HO H OH HH OH OOH OO OOH HH HO OH HO OH OOH HH OH HH OH OO OO O OH HH OH HO OHH HH OHH OH OH HH OH OH OHH HH HH HH HH OO OO H OO-HH H OH HH HO HO HHH HH OH OH HO OO HO HH H HH OH OH HO H OH OH HH OH HH OO HH HH OH HO OH OH OH HH OH OH HO HO H O HO O HH HO HO H OH HH OH OO HO . H H HH OH HO HO O O HH HH HH HO HO O HH OH OO HO OH H OH HH OH OH HH H so: H OH OH HO OH HO H OH HH HO HO HO H OH H HH OH HO H O OH O HO HH OO OO HH H OH HH HH HO OO H O HH OH HO HO H O OH OH HH HH OHH H OH OH OO HH OOH O H OH HH OO HO O O O OH HH HO HO HHH H OH HH HH OH OH OH O HH OH HO HH H OH OH OH OO HO OHH O OH HH OH OO OH O HH HH HH HH HH H OOHHH O HH OH OO HH HH H OH OH HH OO HH O HH OH OH OO HO H HH OH HH OO HH OH OH HH OH OH HH HOH OH OH OH HH HH HH O HH OO HH OH OH OH HH OO HH HH OO HO HH OH OH OO OH OH H O HH HH HH HO HH OH HH OH OH OO OH OH OH OH OH HO OO H IOOOOH OH HH OH HH HH HH HH HH OH O OH OH OH HH HH OH OO OO H OH OH HH HO OH OOH HH OH OH OO HO HO HH HH HH HO OO HO O O HH HO OH OH HH OOH HH OH HH OH HH HH O HH HH OH HO OO H OH HH OH OO OH OO OH HH HH HO HO OH O HH OH HO OH OO H OOxHH OH OH HH OH HO HO O OH OH OO HO HO O HH HH OH OO HH H H OH OH HH OH OH O HH OH HO HO HH HH HH HH HH OO HO O OH HH OH OH OH HO O OH OH OH HH OH HH O OH OH HH OH H HH HH HH OO OH HO HH O OH OH HO OH oH O OH HH HO HO H EOOOH OH HH HH HO OO OH O O OH HH HO OH OH HH OH OH OO OH H o OH HH OH OH OO O HO HH HO HO HO OH HH HH HH HH OH O O HH OH OH HO O: HH H OH OH HO HO HO O OH OH OH HH OO H O OH OH HO HH HH OH OH OH HO HO HO H O OH HH OO HH H OOHHH OH OH OH HO OH OO HH O OH OH HH OH OH OH HH HH HH OH H .uaz .nsm naouu .q.a:w mo ~0>uu mo .02 2 O 2 O 2 O 2 O z a x O x O 2 O a O OaOOz Ho .OasOHOOOH: OH HH OH OH HH OH OH HH OH OOOH .u a. N. n. uco u no Ouaaananou .Ha=z suon cuuuuuu chum-u: non-oauu val noduuunoucm .ququ QEOuH .luun unencuoonn vac Acahoz ozu co vunum .aoHuu-uuunH nous-nu: vuuaoauu an uuuuannm on» now u any we ..noHuum uoHuuuaum ooo~.»o :« .aoHucusvunm HuoHHHaam any .MH vanaa 89 starting point for the number generator the significant differences would vanish. The multivariate analysis of the data in Table 17 based on dichotomous variates indicated significant effects due to: (1) the probability of a one, (2) the number of subjects, (3) items fixed vs. random, (4) the level of sub- ject heterogeneity, the interaction of (l) and (2) and the interaction of (l) and (3). The marginal means for a = .05 regions for the probabilities of a one equal to .5, .2 and .1 were 57.7, 72.0, and 90.3 respectively; for numbers of subjects 4, 6, 8, 10, and 12 they were 55.8, 68.3, 73.9, 86.3, and 82.3 respectively; for items fixed 79.4, for items random 67.2, and for levels of subject heterogeneity 1, 2, 3, and 4 they were 61.5, 70.3, 77.4, and 84.1 respectively. The two interactions are displayed graphically in Figures 6 and 7. Inspection of Figure 6 indicates that a favorable comparison of empirical probability of a Type.I error to nominal a occurred only for 6 subjects and a probability of a one equal to .5, and for 4 subjects and a probability of a one equal to .2 or .1. For all other conditions the test is too liberal. Figure 7 is self explanatory. A multivariate analysis of the data in Table 18 indi- cated the same trends and significant effects that were found in Table 17, therefore only the overall mean vectors for the data based on normal and dichotomous variates in Table 18 will be reported in order to avoid tedious IZO llO IOO U) c o H m o *4 90 c o '3 o o '3; 80 Cd n 70 .5 ‘3 60 '8 a m 5 0* 8 50 m 40 0 Figure 6. 90 Number of Subjects |2==C |O==O 8==- 6 : C] O 4 = * I!" 0, Ht 1 L, l 0.5 0.2 OJ Probability of a One The interaction of the probability of a one and the number of subjects, with respect to the data in Table 17. 91 '00 T I 90 - 2 o m 30 ... C o '3 o m L? tr 7C)- u .5 6C)- § ---- Random % 50.. Fixed u m 4f)- 2? O i J l (15 (12 (1| Probability of a One Figure 7. The interaction of the probability of a one and items fixed vs. random, with respect to the data in Table 17. 92 repetition. For the data based on normal variates the over- all mean vector was 51.1, 26.3, 11.7 and for the data based on the dichotomous variates the overall mean vector was 78.3, 49.8, 28.2. Tables 19 and 20 contain frequencies in a = .05, .025, and .01 rejection regions of variance ratio tests for sub- ject by repeated measure interaction effects, when the data were simulated under the non—null interaction effect conditions indicated in Chapter IV. Relative power variables were formed for both Table 19 and Table 20 and separate multivariate analyses of variance were performed on the relative power variables for both tables. In both analyses significant effects were found due to: (l) the probability of a one, (2) the number of sub- jects, the interaction of (1) and (2), the interaction of (1) and items fixed vs. random, and the interaction of (1) and the level of subject heterogeneity. In addition the analysis of the relative power variables from Table 19 disclosed a significant main effect due to items fixed vs. random. For Table 19 the marginal mean relative powers for a = .05 regions for the probabilities of a one equal to .5, .2, and .1 were .57, .70, and .81 respectively and for numbers of subjects 4, 6, 8, 10, and 12 they were .67, .69, .64, .79, and .69 respectively. The three interactions are represented graphically in Figures 8, 9, and 10. For Table 20 the marginal mean relative powers for 9Z3 am On om. 03a mma ONN NON OO ONN OON NNN NNN NNN NON ONO NNN NNO ONN O NN On NON NO NNN NON NON ON OON NON NON OON NON ONN NNO OON ONO ONN N .833. OO no NO NON ONN NON NO ON NON NN ONN ONN OON ON NNN NON NOO NNN N OO NN NN NO NON NON NO ON NNN NO NON OON NON NO NNN ONN OON OON N OO 90 ONN ONN OON NON NNN On NNN ONN NNN NNN ONN ONN NON ONN NOO ONN O NN O: NO ONN NNN NON NNN NO NN OON ONN NON OON NON NON NON OON OOO OON N OuxNN NO OO NO ON ONN OON OO OO ONN NNN ONN ONN ONN NON NON ONN NOO NON N . OO ON NO OO NNN NON NO NO OON ON NNN ONN ONN ON NNN ONN NOO NNN N OO O0 NO ONN ONN NNN ONN OO OON ONN NNN OON OON ON ONN OON ONO ONN O ON OO NO ONN OON NON OO OO ONN NO NON NNN OON NO NNN NNN NNO ONN N 835. ON OO NO ON OO ONN NO NO. NON ON NNN OON NON OO NON NNN OON NON N ON ON OO NN NON OON ON ON NNN NN OON NNN ONN ON NON NNN ONN OON N O: NNN OO NON NON NON NON NO OON NNN NON NON NNN ONN OOO OON ONO NON O . ON ON OO OO NNN NNN ONN OON ON OON ONN ONN ONN NON ON ONN ONN NNO NNN N Ooxym NO NO NO NO ONN ONN OO NO NNN N NON NON NON NO NON NNN NNN OON N ON ON NO NN NNN OON NN OO NNN NO OON ONN ONN OO ONN NNN NNN NON N ON ON ON ON NON NON ON ON OON OO _NON ONN ONN NO ONN NON NOO NNN O ON ON OO OO NNN ONN NO NN NNN NO ONN NON NNN OON OON OON ONO ONN N aosae ON On ON ON ONN OON OO ON ONN NO NON OO OON NO NON OON NON NON N ON ON ON OO OON NO NO NN NNN NO NON ONN NNN NO OON ONN NON OON N NO OO NO NNN OON NON OO NN OON NON ONN OON ONN OON ONN NON OOO ONN O O ON :O OO NON ONN NNN NN ON NNN NO ONN OON NON OO ONN NNN OOO OON N OeuNO NO ON ON OO ONN OON OO NO OOO OO OON NON ONN NN ONN ONN OON NON N ON ON NO ON NO ON NO ON NO NN NON ONN OO NO NON NNN NON OON N ON NO OO nO ONN NO OO NO ONN OO NON ONN NON NO NNN NNN NON ONN O OO NN OO ON ONN NNN NO ON OON ON NON NdN NNN OO NNN NO ONN ONN N geese ON ON On OO OO OO OO NN ONN NO NON ON ON OO ONN NO NON NON N NO NN OO ON NNN OO OO ON NO OO NNN NO NN OO NON OO NNN NNN N ON ON OON NO ONN NNN ON OO ONN ONN OON OON OON ON OON NON OOO ONN O O OO NO ON NO ONN NNN NO NO NNN OO NON NNN NNN NO ONN NNN NON ONN N ouN .O ON mO NO NNN OON NO OO NNN NN NNN OON OON OO NON NNN ONN NNN N O O NO NN OO ON OON ON NO ON ON NN ONN NNN ON NO NON ON OON NON N NO OO NO NO ONN OO NN NN ONN NO OON NN ONN OO NNN ONN NNN NNN O NO ON O0 ON NNN OO NO ON NON NO OON NO NON ON NON NON OON ONN N aoeza ON ON OO ON OO NO NO ON NO ON OON NO ON ON ONN ON NNN ONN N NO NN NO NN NNN OO ON ON OO OO ONN NO ON NN ONN NO OON ONN N ON NN NO ON NNN NN OO NO OON OO NNN ONN NNN OO NON NON OON NON O O a On NO NO OO NO ON NO NO OO OON OON NO NO OON OON NON NNN N OouNO NN NO OO OO ONN NN on NO OO NO OON ONN ON OO ONN OON NNN ONN N NO ON ON ON O0 NO ON OO OO OO NON NON ON on NNN NO ONN ONN . N .Ouz .OOO ..mmmNNIIl.4mamawuuul NO H0>02~ MO .02 z a z a z D z a z a z n z a z a z n HEOZ BO .3808020HD ON ON ON ON ON On ON ON ON 83.x N. N. N. OOO O No NONNNOOOONN ..Zaz cause-u: van-onus .Saaluoz nuoouuu nowuocuouau 600.95 a: .35 goaouonuan can Héoz 05 co vonnn 533335 .9330: @3123: up 3002.5 05 you w as» we .33u3— guuuoqom 83.3 a.“ .oowuuosvoum Hashing 05. .2 03cm. 94 O OO NNN OO OON OON NON NO OON ONN OON OON OON ONN OOO NON NNO OON O NO. NO OO NN OON NNN NNN NN ONN NNN ONN ONN OON OON NNO ONN OOO OON N 8.25. OO NO ON OO ONN NNN NON NO OON NO OON NNN ONN OO ONN OON OOO OON N O: NO ON OO NNN NNN OO ON NNN NN ONN NNN NNN ON ONN ONN NOO OON M NN NN On NON ONN ONN OON ONN OO OON OON ONN OON ONO OON NOO NNN OOO OON NO NO ON ONN ONN NNN ONN ON NON ONN OON ONN NON NNN ONO NON OOO OON N OONNO OO OO OON NNN OON NON ONN ON ONN ONN OON OON OON NNN ONN OON OOO OON N NO NO NO OO OON NON NN OO NON OO ONN NON OON NO NON NON ONO NON N OO NNN OO OON ONN NON NON NN OON NNN NON NON NNN NON NNO NON OOO ONN O NO ON OO NO OON NNN NO NO ONN OO NON NON ONN NON OON ONN OOO NON N aoaaa NO OO OO OO OON ONN OO OO ONN OO ONN NNN NNN ON ONN ONN ONO ONN N ON ON NO ON NO NO NO ON NNN NO NNN OON OON NN OON NNN OON NON N aO ONN NO ONN OON NNN NNN OO NNN ONN ONN ONN ONN ONN OOO OON ONO NNN O ON OO OO NO ONN ONN ONN NO ON OON ONN OON NNN ONN OO NNN OON ONO NNN N Oourm NO OO NN NON OON NNN OO NO OON NO OON OON NON O NON NON OON ONN N NO ON NO O0 OO NNN ON NO OON NO NON NNN OON OO NON OON NON OON N OO OO OO OO ONN ONN OON NO ONN OO .NNN NNN NON NON OON NON NNO OON O OO OO NO No ONN NO NO NO ONN OON NON NNN ONN ON NNN OON NO ONN N ao_. ON NN NO NO ONN NO ON ON NNN OO NNN ONN OON OO NON OON NNN OON N NOON ON ON NN OO NO NO OO NN NNN NN ONN NNN ONN OO NNN ONN NON OON N OO NNN NO ONN OON ONN NNN NO ONN NNN ONN ONN NON NON NNO ONN ONO OON O O ON ON NO OO OON ONN ON OO ONN OO OON ONN OON OO NNN OON ONO ONN N ON NO ON NO NNN ONN NO NO ONN ON NNN ONN OON OO NNN NNN NON OON N OONNN ON ON NO NO OO OO OO OO OON OO NON ONN OON NO NON ONN NON ONN N NO OO ON OO ONN OO NO ONN NON NNN ONN ONN OON ON OON ONN ONO OON O NO NO NN NN NNN OON OO NO OON NNN OON ONN OON NO NON OON NON ONN N NO ON OO OO OON NO NO NO NNN ONN ONN OON ONN NO ONN ONN NON NNN N aosaa NO ON OO NO NNN OO OO OO ONN NN OON ONN NON OO OON NON OON NON N NO NO NO ON OON OON NO ON OON NNN NNN OON OON NON ONN NNN NNO OON O O OO O NN NO NNN NON NN ON NNN OON NON NON NON ON OON ONN OON OON N OO NO ON NO ONN NO NN OO NNN ON NON ONN ONN NO ONN NNN NNN OON N OOONO OO ON OO OO OON ON NO ON OON OO OON ONN OO N NON OO NNN OON N NO ON OO ON NON OO NN OO ONN ONN NNN ONN OON ON OON ONN NON NON O NN ON ON OO ONN NO NO OO NNN NON NON OON NON NO ONN OON ONN NNN N NN ON NN NN ONN OO NO ON NON OO OON ONN OON OO ONN OO OON OON N aosaa ON ON ON NN OO OO NO NN OON NO NON NO NO OO NON ON NNN NNN N . OO NN NO NO NNN ON NO NO OON NO NNN ONN ONN NO NNN NNN OON OON O O ON NO OO ON NO OO NO OO OON ON NON OON ONN NO NON ONN OON OON N ON ON NN OO NNN OO OO NO OON OO OON NNN OO OO OON NON ONN OON N OONNN ON ON ON NO ON ON NN OO ON NN ONN NNN OO OO OON NO OON OON N No NOOON 2 O 2 O 2 O 2 O 2 O a O z ,O 2 O 2 O ON ON ON ON ON ON ON ON ON N. N. a. O. .332 .3530: v3.2.3 ..Saauaoz nuuouuu uoNuunuouN—H €3.02 .60: .33 39.83503 vac ASE—oz 05 no woman 3030335 .3530: vounoauu an 3002:» 05 you u 05 no .303: 9330?: 82.3 a.“ .3uuausaoum ”cacao-Nu 2t. .2“ v.33. 95 09— 08- - 07F , 06- ./ Number of Subjects Relative Power 05- I2=o 6:0 IO=O 4=* J? 8 =C1 L, L J, O 05 02 OJ Probability of a One Figure 8. The interaction of the probability of a one and the number of subjects, with respect to the relative power of the test for the subjects by repeated measures interaction based on dichotomous data under Design 2. Relative Power 96 0.9 - O.8 — 0.7 F 0.6 ~ 0 5 Fixed ' - — — — Random ‘? O l L l 0.5 0.2 O.| Probability of a One Figure 9- The interaction of the probability of a one and items fixed vs. random, with respect to the relative power of the test for the subjects by repeated measures interaction based on dichotomous data under Design 2. 97 LC)? 0 0.9 - O.8 — I H ’ ‘0 m a) 0.7— > I 'H 4..) m o '3 I m 0.6 P Level of Subject Heterogeneity 0.5 L . | :O 3 :‘D ‘ 2:0 4 =* r C) 1 1 J_ 0.5 0.2 O.| Probability of a One Figure 10. The interaction of the probability of a one and the level of subject heterogeneity, with respect to the relative power of the test for the subject by repeated measures interaction based on dichotomous data under Design 2. 98 a = .05 regions for the probabilities of a one equal to .5, .2, and .l were .53, .65, and .91 respectively, for numbers of subjects equal to 4, 6, 8, 10, and 12 they were .52, .71, .67, .82 and .68 respectively, and for items fixed the mean was .73 while for items random the mean was .67. The three interactions are graphed in Figures 11, 12, and 13. A detailed interpretation of Figures 8 through 13 and the other results of the analysis of the relative power variables for Tables 19 and 20 will not be attempted because the results of the analyses of the frequencies in Tables 17 and 18 have indicated that the variance ratio tests for repeated measure by subject interaction effects are in most of the instances simulated,too liberal. In general it may be observed, however, that higher relative powers correspond to greater "liberalness" in the test of a true null hypo- thesis. Thus the general increases in relative power observed in Figures 8 through 13 across levels of a proba- bility of a one .5 to .2 to .l are in some sense specious. Tables of frequency data for other tests under Designs 2 and 3 can be found in Appendix A. Two tables of correlations between mean squares have been included in this chapter. The importance of the corre- lation between mean squares in a variance ratio test of a source of variation based on variates with a non-zero kurtosis, was indicated in Chapter III. As was indicated, the correlation between a mean square for a hypothesis and the associated mean square error in a ratio of mean squares 99 |.0- Number of Subjects I2 =0 6:0 0.9~ |O=O 4=* I a: 8 =' g 0.8- m w > :3 O m g» 0.7- " 0.6- / I o.5~ ° C) I 1 l 0.5 02 0| Probability of a One Figure 11. The interaction of the probability of a one and the number of subjects, with respect to the relative power of the test for the subjects by repeated measures inter- action based on dichotomous data under Design 3. 100 0.9 - 0.8 - 0.7 - M 0.6 ‘- m 3 o m m .3 4,; 0.5 - Fixed 3 04 _ .. .. .. .. Random C) l 1 l _ 0.5 0.2 O.| Probability of a One Figure 12. The interaction of the probability of a one and items fixed vs. random, with respect to the relative power of the test for the subjects by repeated measures inter- action based on dichotomous data under Design 3. lOl LC)r ' Level of Subject 0.9 r- Heterogeneity | =C> 3:30 08— 2“ 4=* . u m 3 o m o a) 0.7" > .3 - w /’ l’“ w m 0.6 '- O I 0.5 c ‘7 C) .L J .Ji 0.5 0.2 0' Probability of a One Figure 13. The interaction of the probability of a one and the level of subject heterogeneity, with respect to the relative power of the test for the subjects by repeated measures interaction based on dichotomous data under Design 3. 102 was not expected to be necessarily zero, when the dependent variable was not a variate with a normal probability density. The correlation between MSh and an associated MSe in a variance ratio would be expected to be zero only if the kurtosis of the dependent variable was zero and the dependent variables were independent from each other. The kurtosis of the binomial variate is given by the expression: y2 = (1-6pq)(npq)—l, where p is the probability of a one on any trial, q = l-p, and n is the number of trials. The Bernoulli variate is a binomial variate where n = 1. Thus the kurtosis of a Bernoulli variate is (pg).l - 6. Bernoulli variates with parameters p = .l, .2, and .5 would have kurtoses 5.11, 0.25, and -2.00 respectively. If the depen- dent variables in the cells of Designs 2 and 3 were indepen- dent of each other, the expected values of the correlations between various mean squares could be calculated from the above information and the expression for a correlation, 4n2k 2n(nk-l) 2 -% = Y2 [(k-l)(n-l) + (k-l)(n-l) Y2 +Y 3 corr [MS h’ MSe] . However, since the dependent variables in the cells of Designs 2 and 3 were not generated in a manner so that they would be independent the expression for the correlation between mean squares given above is not appropriate. The expression for the correlation between two mean squares for non—independent data involves complicated terms with many cross expectations which are non-zero. No attempt will be made in this paper to represent the expressions 103 mentioned in the previous sentence, but the empirical values of the correlations between mean squares in variance-ratio tests of repeated measures effects and subjects by repeated measure interaction effects will be presented. Although the precise values of expected correlations between mean squares of interest were not calculated, it could be predicted that the correlations between two mean squares of interest would have a rank ordering with respect to the kurtosis of the dependent variable. That is, for three different data simulations where the kurtosis of the dependent variable for simulation 1, y2 , i = l, 2, 3; has i rank ordering y21 > y22 > y23, the rank ordering of the correlations, corr (MSh, Mse)i’ i = l, 2, 3; may be expected to be corr (MS MSe) > corr (MS MSe)2 > corr (MSh, MSe)3 . h’ l h’ Recall that the kurtosis of the dependent variable is a function of p, the probability of a one in the data and observe that the expected rank ordering is borne out in Tables 21 and 22. Tables 21 and 22 are six dimensional arrays of corre- lations with marginal labels. The labels crossed and nested in the far left margin of Tables 21 and 22 indicate respec- tively whether Design 2 (items crossed with repeated measures) or Design 3 (items nested within repeated measures) was the model for the data simulation. The next left-most margin indicates the number of subjects associated with a 104 300.0 030.0 000.. 500.0 00~.0 50~.0 000.. -5..0 .00.0 300.0 500.3: 030.0- eezsm n03.0 003.0 530.0 000.0 050.0 050.0 05..0 00..0 030.0 000.0 330.0: 030.0- 00x00 00 mom.3 931.0 ~00.0 00H.0 N3~.0 000.0 00).; mmn.o cm~.o mmu.ol ~00.nl amH.ol 802:; 003.. n.3.0 .03.» 000.0 000.0 000.0 000.3 00..0 000.0 000.0 000.0: 000.0: 00.00 00 0.00:2. 030... 90.0 003.0 000.0 00~.0 5-.0 03... 05..0 330.0 32.0 0.0.0.. 03.0.. 58:3. .002 0~0.3 003.0 053.0 033.0 000.0 000.0 000.. 050.0 000.0 003.0 030.3: 000.0: 00x00 0 mo:.0 :nmoo :0n.0 mcm.o omwoo ~o~.o 0n~.J m~aoo haN.o m0o.o 000.00 ~m0.0| 80938 0~0.0 003.0 500.0 000.0 000.0 0-.0 300.0 0~0.0 ~0~.0 000.0 000.0 0N0..- 00x00 0 ANn.0 an3oo mom.0 Ohn.o Fma.o n0~.0 o-.0 maooo «Naoo nno.o ¢¢o.0l ~n~.oc BOEEd ~00.0 300.0 300.0 ~00.0 053.0 030.0 00~.0 300.0 300.0 300.0 300.0: 000.0: vuxrw 3 000002 mmn.0 :ao.o n0m.) m:m.0 Num.0 N0~.o m4~.3 mmmoo an.o mou.o 0m0.o m0o.ol 6033i 500.0 000.0 0.0.0 050.0 00~.0 35~.0 050.0 ~0..0 0~a.o 0H0.0 050.0 350.0: 00x00 00 -0.0 553.0 ~03.0 503.0 000.0 00~.0 330.0 000.0 000.0 000.0 350.0: 000.0: 50230 003.0 003.0 030.0 300.0 ~0~.0 050.0 000.0 000.0 500.0 000.0 000.0 000.0: 00.00 00 000.0 530.0 0.3.) 303.0 003.0 3H~.0 350.0 000.0 ~00.0 ”No.0 030.0 000.0 050:0: 0.00030 000.0 nqt.9 003.) mN3.0 030.0 ~n~.o 0HN.3 m-.0 omH.o -o.0 030.0 0N«.0l vuxfim 0 033.0 043.0 300.0 000.0 00~.0 05~.0 000.0 030.0 03~.o .~0.0 000.0 ~00.0u 50250 ~3n.o «00.0 003.0 nom.0 30m.0 mo~.o mha.o mofl.o ofla.o «no.0 Ha0.ul 000.00 vuxrw 0 0.0.0 0.3.0 503.. 033.0 000.0 300.0 000.0 000.0 000.0 330.0 000.0- 050.0- aoEad 000.0 000.0 503.0 000.0 003.0 000.0 000.. 300.0 ~0~.0 330.0 000.0: 300.0: nuxrw 3 3m:.0 on:.d 000.0 )mn.u o~n.o 00~.o onaou m~3.0 HJN.3 000.0 0H0.0 000.0! BOEEd 0.3.0 ~n3.0 530.0 000.0 05~.0 050.0 05..0 0~..0 030.0 000.0 330.0: 000.0: 00x00 NH 333.0 500.0 003.0 050.0 000.0 000.0 550.0 000.0 000.0 000.0 030.0: 050.0: .26:.« 0.3.0 n3..0 003.0 000.0 000.0 000.0 000.0 000.0 000.0 000.0 000.0: 000.0: 00.0. 00 0.00:0. ~5n.o 000.) mom.o Hmn.o H¢~.o mom.o moH.o mNH.o ~0H.0 mqfi.o 300.0 000.0 Bogax .éoz 000.0 003.0 053.0 033.0 00~.0 000.0 000.0 05..0 000.0 000.0 030.0: 000.0: 00x30 0 030.0 000.0 003.0 003.0 003.0 000.0 000.0 0-.o 030.0 000.0 000.0 030.0- eoEae nu0.0 003.0 500.0 000.0 000.0 0-.0 300.0 0~H.o ~0H.0 000.0 000.0 0~H.0c 0uxr. 0 300.0 003.0 003.) 030.0 000.0 000.0 ~0~.0 0~fl.0 000.0 000.0 000.0- ~50.0c aogae 000.0 300.. 300.) 000.0 053.0 030.0 00~.0 300.0 300.0 000.0 300.0: 000.0: 00x00 3 0900050 000.0 030.0 533.0 003.0 00~.0 000.0 000.0 050.0 300.0 300.0 000.0 000.0: 90230 500.0 000.0 0.0.- 050.0 00~.0 35~.0 050.. ~0..0 .~H.0 000.0 050.0 350.0: 00x00 50 003.0 503.. 000.0 000.0 00~.0 350.0 000.0 500.0 500.0 ~00.0u 000.0: 050.0: 50230 003.0 003.0 030.0 300.0 ~0~.0 050.0 000.. 000.0 5-.0 000.0 000.0 000.0- 00x00 00 003.0 043.0 003.0 003.0 500.0 000.0 030.0 000.0 030.0 030.0 050.0 3N0.0u 9:500 . 093000 :nn.0 mn¢.0 nm:.0 mm¢.0 00m.0 ~n~.0 0H~.0 05a.0 0nd.o -0.o 030.0 m~H.0! 00x00 0 003.0 503.0 030.. 0H3.0 50~.0 030.0 350.0 ~0H.0 050.0 000.0 000.0 000.0: 50250 0.0.0 0.0.0 0.3.0 000.0 000.0 00~.0 050.. 000.0 000.0 3~0.0 000.0- 500.0: 00x00 0 53n.0 mun.0 033.0 -3.o mm¢.0 ma¢.o nmm.o anN.o 0a~.o 03H.o 000.0 000.0! 8°95m 000.0 000.0 503.0 000.0 003.0 000.0 000.0 30~.0 ~0~.0 330.0 000.0: 300.0: 00x00 3 MMWUH m.b:m .muuamuo: mswua no .02 noumoaum e 0 ~ H v n N H 3 n N A mw«ocmdmmwuu: uuofinam 00 00>»; H. N. m. 0:0 a 00 500000.00». oouuavm ado: cooauon accuuoaouuoo 050 .‘n .0022 :uon uuuouuu ounces: vuuqoaum vac noduuuuoucH .0010 cacaOuonu«0 no woman .vou-«UOoq4 uouuu «known one: 0:» van consequz voucoaum new oHnuh 105 000.) h)n.3 00.3.0 300.0 3NM.O th.O own.) N3N.O 300.0 0H0.0! 00".”! 33~.0! Eovcmm Nsn.o 3Nn.0 3Nn.o 33m.o 00m.o 0nn.o mNN.0 00H.O m«0.0 NHO.UI 3~N.o! Hmm.0! “Exam NH 3H3.0 H03.o 0H0.0 000.0 3m~.o on~.o mNN.0 dn .0 030.0! 000.0! H3N.O! N0n.o! E350: 300.0 000.0 mm~.0 00H.o ~0n.0 H0~.0 3m~.0 00H.o HMH.0 m~0.0I NOH.o! mon.o! vwxfiu 0H 030.0 HV3.0 n0n.o Hmn.o 3N3.o Hmm.0 0mm.» HJ~.0 mn0.0 000.0! HON.n! 000.0! EOE:& H95:0U 000.0 nvo.0 030.0 000.0 0n3.0 0H3.0 HHm.0 ~0~.o 050.0 3~0.o! 00H.o! 00~.0! vwxrm 0 acoz nun.o 333.) 003.0 000.0 H3~.0 0H~.o 00H.3 00H.0 ~MH.0 000.0! 3HH.u! snn.0! 60352 nmm.0 000.0 300.0 000.0 H~3.0 0mm.o N0~.3 3)~.o >~0.0 3HH.0! m0H.o! 30~.0! “Eme 0 wvn.o N03.3 003.9 000.0 -~.0 0n~.0 00H.0 HOH.0 030.0! 000.0! ~0H.0! m0~.o! BOEEE 030.3 303.3 000.3 003.0 mn3.0 ohn.0 non.) ~n~.0 000.0 030.0! Nn~.u! 00~.0! voxfim 3 000.32 333.3 333.3 330.0 H~0.0 0mm.0 303.0 ~03.0 033.0 030.0 0H0.o! 0HH.0! 03~.0! 60281 000.0 HH0.) 303.3 0~n.o 0~n.0 ~00.o 33~.3 m3H.0 mHo.o mHo.o! Nn~.o! 00~.0! Hixfim NH 000.0 )Mo.u 0m0.0 030.0 H03.o 003.0 ~03.0 ~H3.0 00H.0 000.0 H00.o! m-.0! 50952 nnn.o 333.) dflN.O N3H.o mmn.o 00~.o 00H.) MNH.O NNH.O NH0.0! 00H.0I mm~.0! vuxfim OH Hw0.o 033.0 003.3 Hno.u 003.0 HH0.0 003.0 n33.0 030.0! nHH.o! 3HH.o! 0H~.n! EOEEd Nfln.o non.) 03a.) 030.0 H03.0 000.0 N0N.J n-.o N00.0 300.0 m0H.0! m0~.0! vwam m Huhucflv 300.0 300.0 Hon.) Hmn.0 ~01.0 0~3.0 000.0 033.0 noH.o Hwo.o! NBH.0! ~0~.0! 802:; 300.) 000.3 000.0 0Hm.o 000.0 00n.0 00~.3. 00~.0 000.0 30H.o! 33H.o! mm~.0! voxfiw 0 HHn.u 0Vn.0 030.3 000.0 3H0.0 H33.0 003.0 Hon.o H0o.o 000.0! 000.0! 00H.0! sogad 000.0 033.3 Hun.3 003.0 000.0 500.0 mm~.o 00H.o H00.o 300.0! How.o! 33~.o! voxrm 3 030.0 033.0 300.0 000.0 0H3.0 003.0 mmm.3 00~.0 000.0 300.0 000.3! mH~.o! aoEEd n3n.0 Dao.) 0N0.) 303.0 0H3.o nmm.o m3N.) 00H.O 00H.O )H0.0 033.0! HNn.ol vwxfw NH Han.o 0Nn.0 Nn0.o 000.0 n3m.o m3m.o Odw.) HQN.O mm0.9! HHH.0! m0~.a! mHn.nl BOEEd 0mn.o 3Nn.9 >Hm.n NHm.O 00m.0 00m.0 30N.0 NNH.O «00.0 H3o.0 onH.o! 00m.o! vwxfm OH 030.0 0nn.o mmm.0 003.0 0N3.o snn.o 300.3 nam.o wmo.o 0N0.o 003.0! N3N.O! navcflm Hauucou 030.0 03n.0 0N0.) 000.0 mHm.O 0H3.o Nmm.0 Hom.o NHH.O 530.0 000.0! 03N.o! voxfm 0 !:OZ 0.0.0 9.0....) 3.13.0 H03.0 mHn.O 3)M.O NMN.) HNN.O 300.0 QNQ.OI 30H.o! N0N.0I 509.5% H5n.o 000.0 030.0 000.0 033.0 mmn.o HmN.0 ~0N.0 NMH.O 0H0.o 000.0! nmN.O! voxfm 0 030.0 000.0 003.3 H03.o Nmm.0 Hn~.0 0wn.o 030.0 HHH.0 000.0 00H.o! n-.0! aogzé 030.) 30n.0 mwo.o 000.0 003.0 003.0 ONm.o onm.o 00H.o >m0.o! m00.o! 00N.o! vmxfm 3 vooMOuu 000.9 030.0 Hno.o 500.0 003.0 H03.o «03.3 003.0 00H.o 00H.0 000.0! 00H.0! agflaz £03.00 nnn.0 003.0 0N3.o 00m.0 th.o H03...) HbH.o 00H.O 0H0.0 N003”! NN~.O! 0003.." NH 530.0 310.3 000.0 300.0 003.3 303.0 003.0 003.0 HOH.o 300.0 0>.3! 03H.3! 80280 3nn.0 003.0 003.9 m03.0 00m.o 00N.0 NNN.Q mma.o 03H.o nmo.o hHH.n! o-.o! vwxuh OH 000.0 0V3.) 000.0 HH0.0 H00.0 003.0 303.0 Hn3.0 NmH.o 000.0 000.n! HmH.o! BOEEd . m0n.0 003.) 003.3 030.0 003.3 30m.o mmm.» N3m.o sHH.o 000.0 003.3! 00H.0! vvam m Huuucuu 000.0 hum.) 030.3 N00.o 303.0 003.0 ~03.0 3mm.u H30.0 mm0.o m00.ql 00H.0! 6098i ~5m.0 3nn.0 330.0 000.0 300.0 Hun.0 ~0~.0 00~.0 n~H.0 000.0 330.0! 00~.o! vwxfim 0 m~n.0 03n.0 003.0 ~3m.o on3.o 00n.o 003.0 mmn.o 030.0 330.0 mon.0! NNH.0! 8095i 300.0 onn.0 0mo.o 300.0 0H3.o n3m.0 00N.o n0~.o 50H.o 030.0! m0o.o! NH~.0! voxfiw 3 .MENUH 0.030 coHuum MHMuH 00 .oz IuoucH 3 n N H 3 n N H 3 0 N H huHummmmmouw: uuofipsm uo Ho>wq H. ~. 0. 25 0 No 0320.080 .vuucHoooq< uouuu ouonvm and; «Lu wad aOHuuououcH sou .HHaz :uon .uuouuu nous-no: consonox vac acauuquoucH .qumo mJOEOuOLUHo «nu no 03030 9.30: vouaoaom 00 00000030 new oouoavm ado: 0:0 counuom ncoHuoHounoo 0:9 .00 .23 106 given data simulation. The final column margin indicates whether items were considered fixed and null in effect or random and non-null in effect. The top row margin refers to the overall probability of a one in the data given no interaction or repeated measures effect. The second row margin refers to the level of subject heterogeneity. Each element in these two 2X2X5X2X4X3, six dimensional arrays is a correlation between 1000 pairs of mean squares calculated from simulated data in dichotomous (zero-one) form. The correlations in Table 21 are correlations between the mean square for the repeated measures source of variation and the mean square error associated. The correlations in Table 22 are correlations between the mean square for the subjects by repeated measure interaction source of variation and the mean square error associated. The 480 correlations in each of Tables 21 and 22 were transformed by the "Fisher r to Z" procedure and then the transformed variables were considered as dependent variables in a six way factorial design with six fixed main effects and an ANOVA was performed on the transformed data from each table, 21 and 22, separately. The analysis of the transformed data from Table 21 indicated significant effects due to the probability of a one, the number of subjects and the level of subject hetero- geneity. For Table 21 the marginal mean correlations (reconverted from Fisher Z means) for the probabilities of a one .5, .2, and .1 were .04, .22, and .44 respectively; for 107 the numbers of subjects 4, 6, 8, 10, and 12 they were .27, .24, .24, .21, and .20 respectively and for the levels of subject heterogeneity l, 2, 3, and 4 they were .14, .20, .26, and .33 respectively. The analysis of the transformed data from Table 22 indicated significant effects for all of sources found sig- nificant with respect to Table 21 plus significant effects due to items fixed vs. random, the interaction of probability of a one and level of subject heterogeneity which is represented graphically in Figure 14, and three interactions involving the distinction items crossed vs. nested (see Figures 15, 16, and 17 in Appendix B) which were considered of marginal importance. For Table 22 the marginal mean correlations for probabilities of a one .5, .2, and .1 were —.07, .34, and .55 respectively, for numbers of sub— jects 4, 6, 8, 10, and 12 they were .27, .26, .30, .25, and .28 respectively, for levels 1, 2, 3, and 4 of subject heterogeneity they were .18, .24, .30 and .36, for items E fixed the mean was .26, and for items random the mean was .29. As was expected the correlations increase across levels .5, .2 and .l of the probability of a one (which corresponds to increasing kurtosis) in both tests of repeated measures (Table 21) and tests of subject by repeated measure interactions (Table 22). Also in both tables there is an increase in correlation as the subjects become more heterogeneous. 108 Correlation 0.6 h 05- OK}- (13- 02- Level of subject hetero eneity 0.: - g I: O 2==O O .. 3=t3 4=1k - O.I - -C12- _0'3 P 1 1 C15 (12 OJ Probability of a one Figure 14. The interaction between the probability of a one and the level of subject heterogeneity, with respect to the correlations in Table 21. 109 Other tables of correlations between mean squares can be found in Appendix B. CHAPTER VI IMPLICATIONS AND CONCLUSIONS In general the implications of this study are, that although many individuals have had some success in demon- strating that variance ratio tests based on dichotomous data can be assumed to have probabilities of a Type I error very close to a when the l-ath quantile of a corresponding F- statistic is employed as a critical value to define a rejection region, severe caveats should be issued to poten- tial employers of analyses of variance to dichotomous data 5 in a repeated measures design. The most important warning concerns the use of the quasi-F test when the dependent variables are zero-one data; not only was the probability of a Type I error extremely variant in the data in this study, but the frequency of the quasi-F test based on dichotomous data in any rejection region was always less when a null hypothesis was false, than it was when the null hypothesis was true! Further, the "reverse power" of the quasi—F test based on dichotomous data was not a function of the presence or absence of a confounding source of variation. In the absence of a confounding source of variation the usual variance ratio test of repeated measures based on 110 111 dichotomous data gave a good fit of a probability of a Type I error to the corresponding nominal a given a large enough number of subjects, a number that was somewhat fewer than usually occur in practice. When there was a confounding sOurce of variation which was non-null (as was the case with random items in Design 3) the usual test of repeated measures based on either normal or dichotomous data was inappropriate just as was suggested in Chapter I. The power of the variance ratio tests for repeated measures based on dichotomous data were approximately half the power of the same tests based on normal data, where such tests were appropriate. Although the results concerning the quasi-F were the most startling of the results of this study, the most dis- appointing results of the study were those concerning the tests of subjects by repeated measures interactions based on the dichotomous data. A good fit of a probability of a Type I error to a corresponding nominal a and a set of reasonable power characteristics for the variance ratio test of the subjects by repeated measures interaction based on dichotomous data would have been most useful in the investi- gation of individual differences and as an indication of the need of additional individual difference blocking variables, as was indicated in Chapter I. In order to examine the implications of this study more extensively, consider what the discussion and results in this paper might suggest to an experimenter who would like to do 112 a repeated measures type of experiment and analyze his results with hypothesis testing in mind. It has been sug- gested that it is important for the experimenter to decide whether the experimental response evokers (i.e. items) he employs in his experiment are all of the response evokers of interest (that is all of the response evokers to which he would like to generalize his results) or a random sample of those response evokers he might employ. The above impor- tance results from the likelihood that if the response evokers are a random sample, a non-null source of variation will be associated with them. A non—null source of varia- tion associated with response evokers (i.e. items nested within repeated measures or an items by repeated measure interaction) may be confounded with the repeated measures source of variation in the ordinary analysis of variance tests for repeated measures effects (Chapter I, pages 17—18). Further, an experimenter may be unaware of the above con- founding if he doesn't include the response evokers as a factor in his design (Chapter I, pages 13—18). The above contentions are supported by the results (Chapter V, pages 59 through 64) and are important considerations whether the dependent variables may be expected to have a normal probability density or not. Thus an experimenter who is considering doing a repeated measures type of experiment should consider the nature of the response evokers that will be employed in his experiment and examine for possible con- founding, a design and analysis which includes the response evokers as a factor. If an experimenter must have confounding in the analysis consistent with the design of his repeated measures experiment, he is in a somewhat difficult position with regard to analysis of variance testing of the source of variation with which confounding is present. For even if he can expect his dependent variables to have a normal proba- bility density, the results of this study suggest that the quasi-F test will not have particularlygood properties (Chapter V, pages 76, 78). If, on the other hand, the experi- menter must evaluate responses in such a manner that his dependent variables are dichotomous, the quasi—F test is completely unacceptable. If an experimenter finds that he can expect to have no confounding such as that mentioned above, but must have dichotomous dependent variables the results suggest, he can expect to appropriately employ the ordinary analysis of variance, variance ratio test for the repeated measure effects under the following conditions. Appropriate employ— ment is suggested, when there are more than three response evokers associated with a repeated measure, and when (l) the probability of one is close to .5 and there are six or more subjects in an experiment, or (2) the probability of a one is between .2 and .8 and there are ten or more subjects in the experiment, or (3) the probability of a one is between .1 and .9 and there are more than 20 subjects in the experi- ment. The results also suggest that the above experimenter 114 should expect the power of analysis of variance tests based on dichotomous data to be between one third and one half the power he could expect if his dependent variables had a normal probability density. The practical suggestion implied by the results of this study is, that if the above experi- menter must have dichotomous dependent variables he should employ a larger number of subjects than he would if he could expect his dependent variables to have a normal probability density. The results of this study with respect to the test of the subject by repeated measure interaction when based on dichotomous data, suggest that even a very large variance ratio statistic may not indicate that the null hypothesis is false if there are a large number of subjects and the proba- bility of a one is not close to .5 (see Figure 6). For a probability of a one close to .5, however, it would appear that the probability of a Type I error is only approximately 1.2 times the nominal a level. Since the power of the test of the subjects by repeated measures interaction based on dichotomous data with a .5 probability of a one was greater than half the power of the test based on normal data, the results suggest that when the probability of a one is close to .5 the test may be appropriately employed. If the probability of a one is not close to .5, however, the results suggest the above test may not be appropriately employed. As a practical suggestion, an experimenter who could expect possible subject by repeated measure 115 iJTteraction and who must have dichotomous dependent vari— aflales should endeavor to employ response evokers which will qgive him an overall .5 probability of a one. This investigator has no practical implications or suggestions to present with regard to the correlations pre- sented in Chapter V. As far as he knows, the control of the <30rre1ations between mean squares is not in the hands of the experimenter. The results with respect to the correlations “were interesting, however, and something of which many readers may have been unaware. BIBLIOGRAPHY BIBLIOGRAPHY Bradley, J. V. Distribution-Free Statistical Tests. New York: Prentice-Hall, 1968. Donaldson, T. S. Power of the F-test for nonnormal distri- butions and unequal error variances. Rand Corporation research memorandum RM-5072-PR, September, 1966. Gagné, R. M. (Ed.). Learning and Individual Differences. New York: Merrill, 1965. 265 pp. Greenberger, M. Notes on a new pseudo-random number generator. "Journal Assoc. Comp. Mach.," 5, 163-67. Hammersley, J. M. and Handscomb, D. C. Monte Carlo Method. New York: John Wiley, 1964. Hansen, M. H., Hurwitz, W. N. and Madow, W. G. Sample Survey Methods and Theory. New York: John Wiley, 1953. ‘ Hovland, C. I. Experimental studies in rote-learning theory. V. Comparison of distribution of practice in serial and paired-associate learning. "Journal Exp. Psychol.," 1939, 25, 622-33. Hsu, T. C. and Feldt, L. S. The effect of limitations on the number of criterion score values on the signifi- cance level of the F-test. "American Educational Research Journal," 1969, 5, No. 4, 515—28. Hudson, J. D. and Krutchkoff, R. G. A Monte Carlo investi- gation of the size and power of tests employing Satterthwaite's synthetic mean squares. "Biometrika," 1968, 55, 431—33. Jensen, A. R. "Varieties of individual differences in learning." In R. M. Gagné (Ed.), Learning and Indi- vidual Differences. New York: Merrill, 1967. 265 pp. Lehmer, D. H. Mathematical methods in large-scale computing units. ”Ann. Comp. Lab.," Harvard Univ., 1951, 25, 141-46. 116 117 Lunney, G. H. A Monte Carlo investigation of basic analysis of variance models when the dependent variable is a Bernoulli variable. Unpublished dissertation, Univ. of Minnesota, 1968. Mandeville, G. K. An empirical investigation of repeated measures analysis of variance for binary data. Paper presented at the annual meeting of the American Educational Research Association, 1970. Marsaglia, G. and Bray, T. A. One line random number generators and their use in combinations. "Communi- cations of ACM," 1968, ll, 757-59. Satterthwaite, F. E. Synthesis of variance. "Psycho- metrica," 1941, 5, 309-16. Scheffé, H. The Analysis of Variance. New York: John Wiley, 1959. Seeger, P. and Gabrielsson. Applicability of the Cochran Q test and the F test for statistical analysis of dichotomous data for dependent samples. "Psychol. Bull.," 1968, 55, 269-77. Teichroew, D. Tables of expected values of order statistics and products of order statistics for samples of size twenty and less from normal distribution. "Annals of Math. Stat.," 1956, 21, 410-26. Wadsworth, G. N. and Bryan, J. G. Introduction to Prob- ability and Random Variables. New York: McGraw-Hill, 1960. APPENDICES APPENDIX A Frequency data for variance ratio tests other than those included in Chapter V. 118 a): 0v: as? mmn ooa Noe amo Hmo Foo was “as «mm mos was as) omo ooos pea s was )4“ or) mam :na «H mom cam «ma ans nos nee oho ¢oe ems one age nmo m on; How «am nan no, 50¢ wow H¢¢ nNo mam Hao nos ass «as ¢~m ems mom n~m N ao2:& n:o an 95H HHN «:3 “Ha Ono ne~ oak :nn mmm am: Heo o~¢ Jsk «mm :mm “mo H was «on :»m H90 :00 new mmo oom nan 0:0 «9» Hrs w») «Ha ass was oaa nee s HH aoo on: no) man Ham nmo oma moo «so «he saw an» Hoo aim mmo soo nmo man a aHo own mHu 30: 5:0 no¢ can coo owo 090 :mo aHe ohm Hoo mmo owe omo omm N 68:m uao NoH moH ~m~ woo oxn :~o :mw was NH; Nan mac one hem Hap on mmm 036 H 1:) New «om NM: mom no; mfio non ems Nmo sex so» NNm 0km 0mm NNO moo Neo s 0H) SAN on) mhn ooa oJ: ¢o¢ HHm «:4 aHo cps moo HHa mwk QJm mom poo cam n Ame «cm on» mow mos nan has hon msm om: on nun oos mm“ new mmo ooo mo» N sagas Ham HeH Noe 90H hes mw~ o¢n omm nmo mom Na» ham ekn :Nm «He mm: «on mom H mom mNm ~50 coo moo ago ham :ok «no ooh omo New 00) Hum «no nwo moo mmo a oH NH» mom 9;) so: Omo n¢n on Nam Hco koo 000 one mHo map «so Has no» cos n n30 mom mmw nkn 009 m3: ooh um: 0am own Hmo “so Has 04 com 950 mHo He» N suxsm o.“ smH ~38 sow Hap How Nam vq~ one Hnn one 45¢ «on 0mm Jme Ho: 00m Hoe H :yo :om 900 He on: Hmn ~38 mom Hka son omo one ~49 Hmh aka omm was com s mum mnw NHo aka goo 0;: ~mm can nHo no; moo non mJS can moo ems o¢o mHm m 588 mum NnH :3» mm» mum 0:0 psw r:~ Ham mew mo: one Ni: one New «mm was N ao2:a umn HHH has new Hno ~q~ «on HOH «on omw H40 Hmm m~¢ HNN ohm Hsm coo 04¢ H ana 0H7 he; Han oma Nam «no mmm «so sHo 600 «He «:0 was «so new omo HNa q a non -n oHo nu: Hm) oHn ocm N4; :00 awn om» fine ass mac sma mm” 080 new n sow FHN nok son 0&3 Ho: ::o H~m ans mam 0mm FHm Mko 0H4 was “on 0mm moo H sour“ 1H1 mmH son ANN «so new Non qu Hofl How Geo mmm H¢: sz Ham ohm ass om; H mob 4N~ :mm now 0mm ms“ ¢0m mun awn -¢ mmo Nmm hHs Ham «om HHe one Hos q Hmo osH «Hr Saw mow ¢,~ moo ch «as o:n Nam SN: Has 00. oom moo mam «no m om: mmH 0.0 :mH ¢H~ omw ooq an nun now omn Han ooo oo~ «as am; ¢me smm N Exaem How no N)“ OHH Hwa okH ka HoH no¢ omH qmn mmm mmm mmH mos Hmw Ham oom H :mo Nov 0H) me ~3a ¢Ny Ho» Ho. “mm ma¢ ems mkm JHm :No Hon mms mmo mmm e , o moo ¢H~ ask neN ops mum Hes mmm was 0:: Jam Hmm act moo oHo «Ho Nam mH~ n no: ~oH nJS mow on» man Omn QHN Hmo mum Hoh Hm; own mmm sac mqq one mmm ~ sexes mm~ Hm Nu: nyH nmn ka nHm ¢qH ::. NHN okm 0mm men onH Hm¢ How o¢m mom H arm HoH mes :qH man no. Hmm HmH an am~ now HNM mam ooH NH“ moo mom «Ho q NM, so «to noH oso 34H sh: QHH oHo -H Joe 5mm m“: Ham Nam 34¢ Ame mmm m Hm. Hm on: On was so com a» «a. HHH mom okH mew mmH ans mom mam ~04 N aoEae ooH en Ho. 6; NM; No nnH m4 omw mt no, so mnH om 5mm ohH mHo mo~ H «as mmH H90 maH out o~u Hem aqw ace HJm nkh «Na oon mam mmk -m mHm one q s «a: no mHo HoH mks ha. H04 okH man 9mm mac can Hm: cam Ham NH: «mo on m ohm He ¢nx HHH New 01H Mom OHH so; mHN mom ¢o~ mum ooH No: o~m who om¢ N snarl omH we now so Ham on me“ Hi mom an Hm; :oH NmH no“ Hon NoH NH: wow H .su= .53: nauuH m.s=m a: Ho>wH «0 .02 z a z a z a z a z a z a z a z a z a Hcetoz .o .sonHonuH: oH nN on OH «N on 0H m~ on SOOH.x. H. N4 m. use a we huHHHAQDOum .nuon msosouocuHQ can Haauoz co woman muuufinsm HOu m any «0 .acoHuom aoHuuufium oooH\yo :H .ooHucoavoum HaoHansm ugh .HHaz zuom mucouum mouauqoz vouauaox vac coHuompoHCH .twmmOLU meHH oi.‘ mv ~Hnaa 119 «ON N«O OOOH «HN OOOH ONN OOO ««O OOOH «OO OOOH ONO OOOH ONO OOOH ONO OOOH OOO O «NO NO« :NO NHO OOO OOO OOO OON NOO NNN NOO NNO ONO OHO OOO NOO OOO «OO N EON; OOO N«r OOO NH« OOO OOO O«O OHO NOO «NO NNO HNN NOO NHN ONO «ON NOO ONO N O O ’HO NNN «ON NON OOO OOO NNN OOO «NO HN« HHO HNO OON H«« OON ONO HHO O«N H ONO OOO OOOH NON OOOH ONN NOO NNN OOOH NHO OOOH NOO NOOH NNO OOOH NNO OOOH NOO O NH OOO OON OOO ««O OOO HHN «NO OON HOO OOO HOO NON ONO ONO NOO HOO OOO OOO N NON OON OOO NN« NOO OON O«O ONO ONO NNN «NO OON OOO HOO OOO NON NNO OHO N OOer ONO OON OON NNN ONO HNO NON NNN N«N OOO HNO ON« HHO ONN NON «NO OOO N«N H NOO NON NON NOO NOO «ON NNO HOO ON« «ON HOO OOO OOO NOO «OO NOO N«O ONO O OOO NO: «NO «O« «NO OHO HNO NHN OOO OON NOO ONO NNO NNN «OO OHO OOO NNO N HOO OON ONO «NO N«O NON NHO NH« OOO OOO N«O OHN NHO NON ONO OON ««O N«N N geese NON NOH OON OON OOO Oo: OON «NN OHO «OO «OO NO« OON NNO «HO ««« «ON N«O H ONO «OO ONN NOO NOO OOO ONO NNO OOO ««O OOO NON NNO OOO HOO NOO «OO NNO O OH ONO ONO OOO ON« ONO NOO NOO HNN OOO NON NNO NON HNO NNO ONO NHO OOO OOO N OOO ««N «NO ON: NOO OOO ONO ON« O«O HOO «OO NNN NON NNN NNO OOO O«O ««N N Ooxsm NHN OOH OON «ON OyO O«N HON NNN HON ONO O«N HNN NNN OH« NNN «NO ONN NON H HOO NO« ONN HN« NON NHO OOO ONN NOO «ON OOO ONO HOO NON OOO OOO OOO NNO O NNO NO3 «NO NO: OOOH N«« O«O NOO OOO NOO HOO OON «JO ONO OOO «NO ONO NNO N O«O «HN NOO ONN ONO OOO ONO OOO OHO O«« OOO ONO HON HNO OHO ONN OOO HON N aos:s «NO OON NNO ONN HON ONN ONO OON NNO NON NON OOO OOO NOO ONN OON NON ONO H «NO NH« NNO OON OOO ONO ONO HOO NOO NON NOO OHO HNO HNO NOO N«O ONO ONO O N N«O OOO oOO NoN ONO OON HNO OOO OOO OOO NNO «ON N«O NNN «NO OON HNO ONO N HOO NON HOO OON N«O NOr OON ONO HHO NN« NNO ONO ONO OHO «ON OON ONO NNN N OONNN NHO NNH OOO O«N HNN OON ONO ONN NOO ONN ONN ONO HOO HHN HNN ONO NNN OH« H «:O ONn O«O OHO NNO HOO «NO O.« NOO NHO ONN ONO NNO NON NOO N«N HOO ONN O NNO NON ONO HON OHO NNN ««N OOO «ON ON« NNO ONO «ON O«O OOO OON NNO ONN N NHN NO. «NN HON ««O NHN OHN «ON «ON «ON NOO OOO HNN NNO NON NNO OON OON . N aszam HO, ON «O« OOH ONO N«N ONO :NH NO« NNN NOO OON NN« NON ONO OHO «HN OO« H :NO O«N OOO «H: ONO N«O NNO NN« OOO NNo ONO ONN «NO OHO N«O NNO HOO .NHO O O O«u .ON OOO OON ONO NON NON NNO OOO ON« OOO ONO O«N «OO OOO ONN ONO ONN N O.N OON OON OHN «ON H«N NON «NN OON ONO OOO NH« ONN NN« HHN ONO NON NHN N HJNHN OO« OO. NO« «OH O«O ONN NO« HOH NNO HON OON «ON HOO NON ON« NOO NOO NOO H NON NO. NON OON OON OON :NO OON NOO OOO NNO NN« OON NN« ONO ONO «ON NON O NNO ... ONN HHN «ON HON OOO ONN N«N N«N ONO HNO OON ON« OON NHO ONN OOO N NN« NO. «NO HOH ONO OON ONN ONH ONO O«N OON OHN NN« OON OOO NNO NHN NO« N aoEas :NN OO OO: OO O«« NNH ONN NHH ON: ONH NN« NNN OON HHN NOO NNN O«« OOO H L OON OO. «ON NNN NOO NON HON OON NNN NOO H«N NH« NON OHO NNO OHN NNN NNN O O OOO NNH N«N NOH ONO NNN ONO NNN ONN «NN ONO NNO NNO HOO OON OO« NON OOO N NN« N» NnO NOH OOO OOH «OO OHN OOO OON NHN HNN NN« NNN HNO O«« OOO O:« N OOxHN NNN « N«O HOH ««« ONH OON NOH NO: ONH OO« NNN OON ONN OOO ONN O«« NNO H .uflm .D—Jm QEQuH Q.n_3w HO H0>0- mo .02 z o z a z a z o z a z a z n z a z a HMMmoz no w3050uozuNa OH «N O« OH «N O« OH «N O« OOOH . x. H. N. «. ago O OO NOHHHOOOOON .muao muoEOOO£UHQ was Nmauoz co comma .OOUUnnam you w «nu mo .Nnaz nuon muoowmm Nuance: vmuwmawm was cofiuomuwucH .vonwz mamuH .mconux coauuouom ooofl.«u a“ .ooNOcmsvoum HOUNNNOEm one . . mfime 1220 ONO ONO HOO HN« OOO OOO NOO NNO HNO NON OOO «NO «OO OOO ONO NOO OOO «NO O OHO ONN N...» HNO OON. OO« «HO NO« N«O NNO HNO ONN HNO ONN O«O ONN «NO NNO N 825 OON OON «NO ON. «OO NNO HHO OON ONO OH« OHO OOO ONN NNO «ON NNO OOO HNN N NN« OHH N«O «NH ONN NNN OO« NHN HON OON ONN O«O OOO NHN NNN N«O OHO OO« H OOO NO« NNO NOO NOO OHN N«O O«N NNO NOO NOO NOO «NO NNO OOO NNO OOO O«O O OH NNO NNO NOO HN« HNO OO« OOO NN« OOO OHN ONO HON HOO NHN «OO NNN «NO HHO N OON NON ««N O«N ONO OOO OHO OH« «NO NHO HHO OON OHO OO« ONN NNO OOO OON N OONNN HNn O«H NOO NNN N«N NNN NN« O«N O«O NON O«N NOO O«« NNN NOO NNO «NN H«« H OOO OON HNN HN« OOO OOO N«O NN« NNO HNO .«OO NON NOO NNN NNO «NN NNO NNN O NOO HON ONN NON N«O ONO NNN ONO NOO ON« NOO OOO NNN OO« NOO ONO NOO ONN N ONO ONO HNN «HN ONO NHO OOO NON O«N «ON NOO OOO OOO «OO OON N«« ONN OOO N aosae OON NNH OON HOH OOO NNN OHO O«H NN« «NN OOO ONN NOO OHN NN« OON NOO OOO H O«« O«O «N« OO« OOO ONO «OO NN« ONO NOO ONO OON OOO «ON ONO HOO «NO NON O N ONO NON OHO H«O OOO HON ONN NOO OOO OO« N«O NOO OON NOO «HO NNN O«O OHN N O«O NON OON OON «NO ONO ONO «ON N«N HOO OOO «N« OOO NHO «ON N«« OON HNO N OOer OON ONH «N« ONH OOO HON OON O«H O«« O«N O«O OON O«O NNN NN« OON HOO N«O H «HN O«N NH. OON ONO «ON NON NNN OOO OOO ONO ON« «NO ON« ONN OHN ONO «ON O . N)O ONH OON OON N«O ONN ONO «ON NON HOO NON OO« ««O OHO OON N«« NNN OOO N NOO OOH NOO «ON NON ONN NOO ONH ONO DON NON ONN ON« ONN ««O NNO ««N OO« - N aosza NON «O NNN NNH ON« ONH NON HNH ONO OOH N«« NNN NHN O«H OHO OON ON« OON H NOO ON« OOO NOO NOO NO« NHO NOO HHO «N« «OO HOO ONO NN« «OO HON NNO NON O O NOO ONO OON ONN ONN ONO OOO NNN NON OOO ONN OO« HOO HOO «ON OO« NON OOO N NO ON. N«O N«N «ON ONN NH« OON OOO ONN OON OHO «H« NON N«O OHO OON NH« N OOer NOO NO «ON .«H NOO OHN OON H«H OOO ONN HON OON ONN OOH OOO NNN «N« DON H NON NHH OOO N«H NNN NON OO« HON OOO ONN NON ONN ON« NON NOO OO« OON ONO O ««O ON NN« ONH O«O NOH O«O ONH OO« NOH «OO NON NOO NNN HHO NON OHN NN« N O«N NO OOO «O ON« OO NNN «N NOO H«H NN« OON NON ONH NOO ON OO« OOO N aoEsN N«H N« NNN OO NNO NO NOH ON ONN NOH OON ONH OOH NN N«N NOH «ON NON H _ NN« ONH OON NNN OON HON NOO NON OON OOO «ON ONO HHO ONN OON ON« ONN OOO O O OOO NOH NHO ONH NNO ONN «NO ONH ON« HNN ONO ONN N«O OON ONO NHO OON ON« N NNN NN NHO ONH N«« OOH NON ONH NOO NNN OO« OHN «HN HNH OOO NON NN« NNO N OONNO OOH O« O«N NO ONN OHH «OH «O OON ONH OHO OHN HNH OOH ONN OOH HHO NON H .nnmajaflmfilllmmmuH O.O:m mo No>oq mo .02 a O 2 O 2 O 2 O 2 O 2 O 2 O 2 O 2 O Haauoz No OOOsOOOOOHO OH «N O« OH «N n.O« O« OOOH.u . OOO O No NOHHHOOOONN .uuca unasOuozuNa can Nuauoz no venom .ouuonnsm uOu m ecu no .u:0«u0¢ aowuuonom OOOH;Xv aw .ooNucoswoum HaoNuNaam och .«v .Hasz ouoouum cowuuauoucH .Haacvaoz mucouum change: vuuaomox .vomoouo mauuH manna 121 h m «H m mm ,r mH NH pH mm mm H a m ON ON mm OON .N. NH HH mH Ow my HO N 0 HH NN ON on nH HH «N mm «O Hm n mH N OM N N» 0H NH .NO ON ... mm NO N m OH nw mm an N seven“ NH NH 0H uH an em HH OH «N n n Om m mH n~ o aO mm H HH m OH NH oq nw 9H m Hm mm a NO m 0 ON HN 0O OO O NH 44 H OH H Hm ow O m mH nH Nm Nw 0 NH m~ NO Hm Om m NH O. OH OH Nm 54 a a HH hH mm m > 0H 0H 0H an .3 N wax: H. O :H «H Hm mm m 0H NH ON 0: OO NH mH 0H NN mm 0O H «H N NH OH on ON NH O OH OH mm 0O N OH nH mm mm m: O ~H n mH a Nm m OH 0 O NH On an O c N 0H ON a m NH m OH O Hm 0m HH m NH ~m mm on N m mH - on NO N soESm nH HH mH ow w: On m m N m «N «O mH m ON mH mO nO H OH m Hw O NO Hm HH NH OH OH Nm mm NH 0H mH Om OO ON O 0H m m n « Ow O o O NH OH Hm NO HH OH mm mm HO Hm m mH m mH OH on mm 2 .OH HH )N Om O: m 0H H~ mm N4 nm N “5...; o m 04. w. 4n H n 0 NH «H mm mm OH O NN rH hm h: H n 0 HM w on HH m 0 0H m On mm 4 m Hm NH PM 04 .N N .O mH :H ow OON 0H J “N o :4 mm OH .H mm .ON mm mm m m ON NN 0H mm mm m h pH 0 HOO mm mH 0H 4 nN 0: OOO N aovcwx o H oN 1 H4 m“ H. mH ON ON 4: No OOH nH 0N NH mm NOH H m o cu O NO HO OH HH n nH No mm m OH «H NH mm 9O O m m m VN m H OH o «H «H Ow HO Nm O NH H Ow On m m m N NM 0 an «H 4 o Hw OH NO mm HH mH mm mm mm mm N vux: a 0 OH «H nN o: u HH 0H 0, rm cm .OH OH om HOH n4 cm H H. OH «H 0H 0“ mm OH 0 NM NH «3 wm mH h 0m Hm mO on O NH 0 mm N, 91 NH mm H. 0: ON 00 «OH 0H m an mH mm MO m n N OH NH mm «W OH m on H~ no ~m HH m mm mm mm mm N EOEEm 0‘ HH Hm «H mm my O OH an «H ~o OO HH m mm N NO NO H mH N 0N N Nn mH mH m 4m n :n .NH 0 4 mm MH m4 COO .H 0 MH 0 mm 3 NA DH NH O um nH om Hm N O mm ... m: NO m m 0 0n m or 1H HH 1 4m «H n mm m OH mm H. On On N 3wa NH m Ow HH HO OH O m NN NH ON ON HH NH ON Hm Om H H NH HO 3n m m: 1.. HN m cm HH mm um o N ON ON Hm NO ON om o mN n 51 mm OH O mm HH 0: mm o n u mm 0 TO M mH N ou N H: On «H O ON NH m4 ~O N HH ON ON mO Nm N aoEix N: N mm m. on OOO. N N AN ow 04 Nm OHH 0 ON ON 4.. cm H _ ow N Hm m NA N NH ~ N~ N m: ON O N OH om ON NO O O NH 3 On 0 O: NH NH 0 n~ « NO Om HH 0 ON Nu n: «O n ONH o mu 0 a: mu 0 H ON m on ow o H. OH ON «4 4m N ~5me NH O HO m On OH om o m: n NN mm mH NH Hm o Om NO H nun nzm MEUHH m 217 we Hw>uH we .02 z n z o z a z a z o z n z n z o z a 2502 no a I. 3 mm on 0H 2 on 0H 3 ow 003.! N. m. uco m we NHHHHnOAONH . HHHHz zuom wuuuwwm whamuwr. qumwnux vca coHuuuuoucH vmnnn :OHuu-nuucH .355me uwunomwm .3 man: 33 new NH us“ we .nconoz coHuuufium oooHLO :H .nuHucusvo; HauHNHnEm wFH . .3302 3.5: $25 9.553933 vac H5307. 05 co OHOON 122 73 OOH n2 3N in an is EN H2 in 2; n: 05 ..Nm 93 O3 NNN c3 O H3 NNH 3N HNN 3: can ONO SN 23 non omN 3O 23 ««N can Zn 92 H? n 33 NHN 3 can oi. NNO :3 EN :3 3n NON n3 H3O HNN Hmm Nmn m3 34 N EH25 in «.3 O3 NHN 92. SO 3m 93 .30 man N3 of. HNO SN 2m 3m :2 of H N o 2 O NO Hm O N 3 «N Na Nn NH 2 Z 2 T. .3 O NH 1 0 3 H H: “m o ..H “H NH on NO a «H E - Hm .3 n «. OH 3 OH R a N O 3 oN NN u NH o oN ON an on H van: m n Nu OH HO NO o O 3 HN an NO H. m 2 NH 3 «O H a HHH Hm OZ 2 3n m 03 3 0O“ 3 3O o 03 ow Hon 2 3O O O :H on n3 2 9% H mON on Nnn on NHO 9 S: NH omN oo in n 1 SH mu 9: NH 3m 0 RN Nn man an «HO 3 2: on OmN 3 Hum N .825. n H«H Hm HHN H... KN O N2 on 3m 2 N? o m: 3 m3. 3 Nan H H 0 Ha H. OH H H m 3 2 0 mm 2 N on NN no Nm O S H a 2 E on 2 O 2 NH ON On Hm 0H 0 ON 3 3 on n a Z 3 NH 3 3 0 3H ON 3N HO NO o O 2 NN HO .3 N Han: « o .3 2 N0 an N NH NN On No 3 o a Z Hn 3 3 H N HA 2 m4 ON OHN o OOH NH 02 3 an m o: 3 O«N 3 02 O n 3 ON OHH an ONN N “NH 2 SN ON Zn o 12 an RN 3 in n .. No NN NOH “N EN 0 a: H: ONN ON NHm m 2: mm OON 2 can N .52.: N HHH 3 3H 2 OHN m 3H 2 HNN Om NHN nH NOH on SN S 02 H H H. «H N 3 o O 9 HH 0 3 HN OH 2 ON 2 3 3 O o 1 o ’N a an 2 n o “H O E 3 2 2 3 aw 0O NH n N ) 3 OH 3 HO n a NH MN 3 3 0H. m ON 3 3 5 N you: n H, 2 n 3 HH 0 n 2 2 ow MN 9 HH ON ON on on H H 2 0H 3 an 0.; O Om 2 OOH on EN 0 No 3 EH on NmN O 3 3 2 Ho Om H2 m 3 0H NHH Hm H3 a 2. «N :2 3 SN n o 3 Hm NH, NO ONH 0 3H 3 0: 2 3m m 2. NH 3H on NON N EH33— 3 ON 2 NHH on oHN “H 3H o~ 3H 3 RN 0 3 3 T; On OHN H 3 H NN n f O m H 2 a NO ON a: m ON 3 HH mm O o N, a HN H an NH 0 O NH HH R mm NH N ON 2 NO 3 m n O 3 NH NO on NH H S. m .3 NH o m NN ON on Nm N 3:: H. O 2. NH NH 2 c O ON 2 HO 3 N o H: 3 mm 3 H N N Na Z No 3 n 3 Nm on 3 3H m NO ON 3 3 HS O o a 3 Z HO 2 m 2 Hm NO HO 3H O HH 2 H an 1: m o 3 mm ON N: 3 HH on 3 No OO 8H H. S NN 3 3 3H H 585; O 3 3 u 5 Ho H 3 Z 3 NO 2H m R on 3 an HHH H n a I u on n 2 o 3 a 3 UN . m m Z Z 3 an O O O N ) 3 H H5 OH «H m HH 3 2 3 NH HH ON NN 3 On H H, H on H NO 2 2 O OH ,H G 2 a Z mm HH 3 Nm H 3.: o N OH. 2 3 3 HH N NH 2 On an N 0 ON 3 NO on H do: .35 953 ..pnm mo H35.— mo .02 z a z a z a z a z a z H. z a z a z a 2:02 no 4 E 0H 2 on H: 2 cm 0H 3 on 82 .5 H. N. n. 2.5 a we HH.—ISLE . 2:3: Avon nuuowum 253.0: vouaonum v5 aouuuauuucH .vonnono use: 025 0:330:39 van Honz a...» no vooan .33» new m 05 mo .uuonuu H.330?»— oooH..6 nH .nuwocuavoum HGUHuHH—au 05. .. ‘ 0.3:. HOH OON ONN OON HNO NNO OON HON .NNO O OOH OON ONN HNN OON NOO . NOH OON NNN N aogze NOO ONN OON NON OON NOO HHN NON ONN N OOH NON OON NON NON HOO OOH ONN NON H o ~H On m oH on n - me O oH O ON .3 O OH on O OH OO N 3me OH OH oO O N ON O ON OO N OH ON NO O ON NN HH ON OO H OO NOH ONN NNH OON NON OOH NON ONN O ON ONH OHN NOH OON OON OOH OHN OON N EOE; OO HNH NON NOH NON OON OOH HOH NN N NOH HOH OON ONH OON OON NHH OOH HON H N N NN 0 HH NN OH ON OO O O o N OH N OH NO . O OH HO N N O ON O oH HN O NH ON N OOan N NH HO OH ON OO O OH HO H ON OO HOH OOH OOH NON OO OOH OON O QO OO OOH OOH OOH OOH NHN HOH HOH OON N O «O NN NOH ONH ONH OON HON NO NNH OON N aoEOO 1O NO ONH ONN OHH ONH NON OO OOH OHN H o O NN N OH NN O NN O O O o H OH O ON NN O .ON NO N N NH ON 0 HH N O OH OO N Oesm N O OH O OH ON NH ON OO H N OH OO ON OO ONH HO NO OOH O HH NN OO OO OO NH ON NN OOH N OH NO HO HO NO OOH NO NO OOH N aoEaO OH 0O OHH OO NO OOH NO 00 ooH H O O N O H N N NH NN O O o N OH H O o O OH .3 m N N OH N O NH OH OH NO N OOwa O OH HO O OH ON O H OO H .OOO .Oam mmOOH O.O:O NO HU>01~ MO .02 OH ON ON OH ON ON OH ON OO OOOH wOwnul A. , N. m. one a mo auHHHnwnoum .Hasz cowuuauoucu .Hascucoz muuauum whammoz kumuawm .uunnouo uEOuH .auaa asoEOuono«a onu no woman .naouH new m ozu uo .oaonom :oHuuufiom oooH.x.:H .mmaucmskum HmoHuHaam mph . wanna II4IIIIIIIIIIIIIIIIIIIIIHHHHllIlIIlIIIIIIIIIIIII------------------ 4. -, A 1... .Il.l all-l?!- IIllllI-I‘Al 7 124 OOH OON NOO NOH ONN OOO ONH NON HOO O HO OHN HOO ONH NOO NNO OOH OON OHO N 902:; HNH NON NHO HOH NON NOO OO OON OOO N OO HON NNO OOH ONN OOO No HON ONO H O HH OO O NH ONH O HN NOH O NH O O NO H NH OOH O N OHH O OOOHO O O NNH N NH NHH O ON ONH N O NH HHH H NH OHH N NN OH H O ON ONH O ON ONH O ON ONH _ OOOH.OO HO HOH NOO HOH OON OOO OO NOH ONO O OO OOH HOO NOH NON OOO OO NOH NOO O 502:; NO NOH ONO NNH OON OHO OO ONH NOO N NO HOH OOO OO ONN OOO NO NNH OOO H O HH NO O OH NHH O ON ONH O OH O OH ON O OH ONH HH ON NNH N OaxOO O OH OOH O ON NHH O .N ONH . N O ON N NH OH OOH O HO ONH H O ON NNH O N NNH O ON NNH _ OOOH..o ON OOH NOO NO NNH NHO N HNH OHO O OH NO OHO OO NOH OOO OO OOH OOO N ON OO HHO OO HOH NOO HO NNH OOO N a09:g ON OO NON NO OOH NOO OO OOH OOO H O OH OO N ON OO O NH OO O O O O OO O ON NHH O ON OOH N HH OH OOH O OH OO O ON NHH N OONON O OH OO O NH NO O OO NOH H O NN NNH O NN NNH O NN NNH H OOOH .a NO ON NON ON OHH OON OO OHH OHO O NO NO ONN ON HOH OON ON ONH NHO O NO NO OON OO HOH OON ON OO ONN N a02:& OO NHH HON OO ONH HON OO OOH OON H O O NO H HH OO O ON ONH O O ._ N NO O OH OOH H NN OHH N - N O OO O OH OO O NO OOH N OON; O NH OO O OH OO N ON OOH H O ON ONH O OO ONH O ON ONH _ OOOH .3 OH OO HON NO OO OON ON ON OON O ON ON OON O: OO HNN ON OO ONN N . ON OO OHN OO OOH OON ON OO OON N EOEEO OO NN ONN OO OHH OOO NH ON OON H N O NN O HN HOH HH , NO OOH O O O NH N0 O ON OHH O ON HOH N HENHO H OH ON N ON NOH OH OO NOH N O OH OOH O ON ON. H- OO OOH H LIIII. . O NO OOH O NO OOH O NO OOH OOOH :5 .OOO .OOO meOOH O OOO Oo HOOOO O; .32 L H. 1 N. m. wso m we NuOHNLNOONN .HH3z :Oom muuuuum wuzwawz vmuqoamm was coHuomuwucH .floqnuz macaw .eumo m5060u050No onu co vommm .uawuH pow m wcu mo .mconox :oOuuufium ooo~.xv cH .moHocmszum HmuwuHaEm any . O nanny \ 1O255 OON OOH OOO OON ONO NOO NHN NOH OOO ONN OOO OOO OON ONH HOO NON ONN HOO O ONO HO NNO OHN ONO HOO ONO ONH NON OON OHO NNO ONO NOH OOO OON OON OHO N HNO HNH OON NON OOO NHO NNO HOH OOO NON OOO NOO OON OO HNO OON OOO OOO N aogza ONO OO OON HON OOO NNO NNO ONH OOO ONN HOO OOO OON NO NHO HNN ONO ONO H HH O ON HH OOH OO O O NN NH NO ONH O O ON HN NHH NOH O HH O ON O NO NO O H ON NH OO OOH N O ON HN OOH OHH N OH O OH O NN ONH O N HN NH OO NHH O O NN ON OHH ONH N HaxHO HH O OH NH NO HHH O H NN NH OO OHH O N OH NN NOH OOH H .um: .mww mawun No Hu>aq 2 O .2 O 2 O 2 O 2 O 2 O 2 O 2 O 2 O maosoOOOuHO no Hmepma O ON ONH O ON ONH O ON ONH OOOH H. N. O. mam a No Oummmmmmmmm .ouoomnam 0>H039 .Haaz anon uuuuuum okanau: vouuunom vac coauouumucH .vuumoz naouH .uuan QSOEOuoguao cad anhoz no woman .naouu new b any mo .uconom cauuuofium oooH nu .ooauaosduum Haouuaaam «£9 a.. wanna OOO OOH OON OON ONO ONN OOO ONN HON ONN NON NNO ONO ONN OOO OOO NNN OOO O NOO NNH ONN HNN NON OON ONO HON OOO OON OON HOO NOO OON OOO NON OOO HOO N ONO NHN NON OON OON NNO OOO OON NOO OON NON NOO OOO ONN HOO NON ONO OOO_ N aszaN ONO OOH OOO NON ONN OOO NOO OON OOO NON NOO OOO HNO NON OHO OON OOO OOO H N O NO O NO HO O N OH ON NO NO NH NH NN NN NO OO O O O ON ON HO OO O OH OH NH ON NO O NH NN NN HO OO N O OH OH OH NO ON N O OH ON NN ON NH O ON ON OO OO N OesN N O NN OH HO NO O . O OH HN NO NO O O OH NH OO NO . H .uw: .nzm mEUOH mo Hw>oq ,llllllulll'l 2 O 2 O 2 O 2 O 2 O 2 O 2 O 2 O 2 O OgosoOOOOHO ho HOeOoz OH ON ON OH ON OO OH ON ON OOOH .uo H. N. O. OOO O No NOHHHOOOOON. .OHOONOOO m>HO:N .HHOz nuom OOOONNN ONOOOOz OOHOOOON OOO OOHOOOOOOOH .OOOOoHu OaOOH .OOOQ OsoaoHoguHa OOO Hasuoz may no OOOOm .anHH New m oz» mo .anHmOm :oHuOOnoz oooH.xo OH .OOHOOOOOONO HOOHNHOam NON . OHOON APPENDIX B Empirical values of correlations between mean squares in variance ratios other than those presented in Chapter V. 126 04(- 03 — c o -H 13’ H 0.2— 2 g Nested O.| - ...... Crossed C) I i Fixed Random Figure 15. The interaction between items fixed vs. random and items crossed vs. nested, with respect to the correlations in Table 21. i (14-r S 0.3— :3 m 3 H S 0.2- 0 if Nested "°'- Crossed O .4, l I l I I 4 6 8 IO l2 Number of Subjects Figure 16. The interaction between the number of subjects and items crossed vs. nested, with respect to the correl— ations in Table 21. 127 CHSF 0.5 -— I I], ll’ // 0.4 " ’4’ // I’ll, I’ ()3-- 1” c ” o ‘4 .H 9 a: 02- 9 E ,’ u 3 x ,’ OJ - / I ,’ I Items I] I /' I ""' Crossed - Fixed . C)— /’ ———— Crossed - Random ; / I, ' I "-- Nested - Fixed i “(1" ’ .... Nested - Random 3 .412 1 1 n (15 (12 (1| Probability of a One Figure 17. The second order interaction between the probab- ility of a one, items fixed vs. random, and items crossed vs. nested, with respect to the correlations in Table 21. ]-2i3 «No.0 mo¢.o Nam.o OHmOO cowoo oOMOO hcm.o QONOO cho.o mmo.o Om0.0I ”no.0! aoYhi mmc.o OmOOO OHO.o NNOOO oom.o mo~.o cowoo ~ha.o ONHOO n~o.o mmo.o oHfl.ot vSSm NH Nmmoo mnm.o HNm.o homoo QNMOO menoo MOMOO Hmm.o ocooo ccowo coo.o m~0.0l BOYEm NomOO onm.o mn¢.o mmmOO 0mm.o Hom.o mowoo Onwoo ~m~.o ocooo .~¢0.0l m¢0.0I umxfim . 0H Noo.o ocm.o oqm.o omcoo NNOOO mom.o Oo¢.o Hw~.o >0~.o «no.0 oHo.o Ooo.o .aoYEm HOOOQ Ham.o MNmoo omcoo NNOOO memoo ¢-.o OOAOO oofioo mcooo owooo o¢~.0| vmxrm m nwm.o mam-o OOm.o woOOO moe.o nmm.o hmnoo cbwoo ooH.o MNNOO Noo.ol_ ~m0.0l O5Ocn% 0:0.0 omo.o QOmOO om¢.o -m.0 HNOOO oowoo so~.o omo.o «No.0 mmoool OOH.oI vuxrm o oaw.o aoo.o oan.o mnmoo mom.o h~m.o mmcoo ~mm.o hnN.o omo.o m:0.0| ~0~.ol aoE:£ omN.o oo~.o omo.o OOOOO “Om.o h~:.o ~m¢.o ~mn.0 NOH.o «no.0 omo.0I oo~.ol noxrm O . .s--. -.1. .+. OEmOH O.OOO mo .02 O N N .H .N n N a O N N H .. JOHOOOMOO..OO: meflnsm No HOE: H. N. - m. oco c No NONNNLOOOOO hp aaouu an ouoonaom mumsvm ccox «Lu can coauowuwucw mousnmmz vuumomwm,hp namuH oumavm cov: ozu coozumm Ocofiucfimuuoo ~59 l .aasz LOGO mucouum onaomwz.voucuamm can OONuuauuucH .muca maoaouozuua ocu co comma coNuomuoucH owuzmmmz vaumuaom .Nm OHOON 129 ONN.O OON.O HNN.O ONN.O ONN.O NOH.O NOH.O NOH.O OOH.O OOO.O OHO.Ou OOH.O- OOOSO HON.O NNN.O OON.O OON.O OON.O ONN.O HON.O NNN.O OO0.0 NH0.0: NO0.0u ONO.O... 3me NH HOO.O ONO.O OOO.O OOO.O ONO.O NNO.O OON.O ONN.O OOO.O NHO.O OOH.O: OHN.O... OOOOSH OO0.0 OH0.0 OOO.O NO0.0 HOO.O OOO.O NON.O ONN.O NNH.O OO0.0 OO0.0: NHH.Ou 35 OH OOO.O ON0.0 HO0.0 ONO.O OO0.0 OO0.0 OO0.0 HNN.O OOH.O ONO.O.- NOH.O.. OON.O: aochm NHN.O HOO.O OOO.O HN0.0 OOO.O HON.O OON.O OON.O NOH.O OOO.O OHO.O H36: 3me O H582 OOO.O OOO.O OOO.O HOO.O OO0.0 OOO.O OHN.O ONN.O NOH.O NNO.O OOH.O: OON.O: OOOOSH OOO.O HON.O HO0.0 OO0.0 ON0.0 NO0.0 ONN.O NNN.O HOH.O HO0.0 NO0.0.. OOH.O: .3me O OON.O HOO.O OOO.O OOO.O OOO.O OOO.O OON.O ONN.O ONN.O ONO.O OOO.O: ONN.O: 38:3. NNO.O OON.O ONN.O OOO.O OOO.O OOO.O OOO.O OHO.O OOH.O NNO.O OOO.O: OON.O: 35 O ONO.O NOO.O ONN.O ONN.O OOO.O OHN.O NNN.O OOH.O NOO.O OOO.O NNO.O: $0.01: 82:; OON.O ONN.O OON.O NNN.O OON.O NNN.O OOH.O OOH.O OOH.O OOO.O HOO.O ONO.O: OON: NH ONN.O OOO.O OHO.O NOO.O NOO.O OON.O OON.O HOH.O OOO.O NOO.O NOO.O.. NNO.O: OOOOOO NNO.O OOO.O NNN.O NON.O NNN.O HNN.O NOH.O OOO.O NNO.O ONH.O OOO.O ONO.O: OON: OH NNO.O NNO.O HOO.O HHO.O OHO.O OOO.O NNN.O OON.O OOH.O NOH.O OOO.O NNO.O OOOOOO . w commouu OOO.O ONO.O ONN.O OON.O NHO.O OON.O NHN.O OOH.O NNO.O OOO.O NNO.O NHO.O OOOH: ONN.O HOO.O NOO.O HOO.O OHO.O OON.O OON.O NNN.O OHH.O HN0.0 OOO.O: HO0.0: OOOOOO OOO.O OHN.O OHN.O HON.O NON.O ONN.O OON.O OHNOO OO0.0 NOH.O NO0.0 NN0.0: OOOH: O HNN.O OOO.O OOO.O ON0.0 NOO.O ONO.O NNN.O ONN.O OOH.O OOH.O NNO.O NOO.O... 38:2 . ONO.O ONO.O OOO.O OON.O NHO.O O.ON.O ONN.O OON.O A .210 NNO.O OHO.O.. ONOOO- HHH: OI . mawuH m.n=m msouH . No 62 O n N H c m N H q n N H Huwmmwmwuwumm you paw Vb Ho>wa N- O. OOO «No DHHHOOOBO .HH92 sued auchwm ouamuoz voumoaom and coauunuoucu .ouuo .30-Ouoguaa osu no woman .vaucuuooa< uouum oudsvm doc: osu via aflouu you nouasam chat any auwauom mcoNuoaouuoo one . N anmH 130 OOO.O HNN.O ONO.O. ON0.0 OOH.O NON.O OOH.O ONN.O NON.O: ONN.O: ONO.O... NNO.O... 825. ONO.O NOO.O NNO.O. NOO.O NOH.O OOH.O OOH.O NNH.O OO0.0: OON.O: NNN.O.- ON0.0: Han: NH OO0.0 OOO.O 0.3.0 33.0 615.0 #26 93.0 02.0 30.0! ~n0.ol 73.5.. Eaton nova; NNO.O OON.O ONO.O OOO.O NHN.O NON.O OON.O OOH.O ONN.O- ONN.O: NOO.Ou ONO.O- HOSE OH ONO.O NNO.O NOO.O ON0.0 ONN.O NNN.O OOH.O NOH.O OOO.O: OOO.O: OO0.0: OOO.O: nouns. OOO.O HOO.O OON.O OOO.O NON.O HOH.O. OON.O OON.O NHN.O: HON.O: NN0.0: OH0.0.. OOOH: O H.325 OOO.O OOO.O OON.O OOO.O OOH.O ONN.O OON.O OOH.O OO0.0... OOO.O: ON0.0: ON0.0: OOH.O; NN0.0 ONO.O NOO.O NNO.O OOH.O OHN.O NOH.O OOH.O OOO.O.- OO0.0: OO0.0: NN0.0: 3me O NOO.O HO0.0 NH0.0 OOO.O NO.N.O HON.O HNN.O OON.O HNN.O: ONO.O: ONO.O: NOO.O- SOOBH NO0.0 OOO.O OOO.O ON0.0 NNN.O OON.O OON.O ONN.O HN0.0.. NON.O: NOO.O.. NOO.O| .5me O HOO.O HHO.O NNO.O OOO.O NON.O ONN.O HNN.O OHN.O NOO.O.. HNN.O: NON.O: OONMOu OOOOOHH ONN.O OON.O OON.O OON.O HOH.O ONH.O NNH.O ONH.O OOO.O- ONN.O: OON.O: NON.O: 3me NH ONN.O NNO.O OOO.O OOO.O NON.O OON.O OON.O OON.O NON.O: NON.O: HON.O: OOH.O: 82.2 HNO.O NON.O HNN.O ONN.O ONN.O HOH.O ONH.O NOH.O NON.O: NON.O: OHN.O: OON.O: 3:: OH HOO.O HOO.O OOO.O ONO.O OON.O OHN.O NON.O HON.O OHN.O: OON.O: OON.O: OON.O: songs. OON.O OON.O NNN.O HNN.O NNH.O OOH.O NOH.O NNN.O OON.O: OON.O: HHN.O... OON.O: OOxHO O OOOOSO OON.O NHO.O OOO.O OON.O OON.O ONN.O OOH.O OOH.O OON.O: OON.O: OOH.O: ONH.O: 335s . NOO.O ONO.O NON.O OON.O NOH.O NOH.O OOH.O OOH.O ONN.O: OON.O- ONN.O: ONN.O: OON: O OHO.O NON.O OOO.O NON.O ONN.O NNN.O NON.O OHN.O ONN.O: OON.O: OOH.O: NOH.O: SOOOO HON.O NNO.O OHO.O NNN.O NNN.O . ONH.O HNH.O OOH.O OON.O: OON.O: ONN.O: OON.O: 8me O maouH m.n=m mEouH No .02 Q n N A e n N H e n N a uaoco cacao: uuo 23m «0 ~o>vg - N. O. 25 O No NOHHHOOOSO O \ .Hdsz :uom uuuuumu vacuum: vuuaomom vac coNuomuuuc~ .mumo Oceanuonuwo onu co venom .vu»a«00un< henna ouoscm duo: on» was cuooqnam uou mouuavm sou: onu cooauom naOHuaauuuoo mph . wanna 131 mnnoo 0:3.0 choo One-o :Hwoo ONN.O NNN.O 0Fuoo fiance! -nool 00H.OI mm~.0l abgifi 0H mmm.0 O¢M.O Janoo macoo moo-o FNOoo “00.0! 000.0 OFMool ONN.O! NNN.O! Hm~.0l voxflm annoo 0nmoo nmcoo nhaoo onaoo ¢a~00. NNN.O onaoo ¢h~col FONool HnN.OI OON.O! 60258 0 ~m~oo mnwoo omwoo thoo 050.0 NHN.O 050.0 NQOoo nNnool ¢m~ool 00~oOl ChaoOI voxfiw honoo :fimoo ON«.O wan-o ONN.O moaoo MJHoO onaoo OFN.OI ¢hNool NNN.O! od~.0l BOEEQ. 0 Nnnoo 0Mm.0 mmm.0 aNm.o who-0 #nooo 000.0 O#0.o Honool ncn.0l nmN.OI m-.0l flour“ oom.0 :mm.0 Hsm.0 :sn.0 on«.0 aNHoo ¢nH.0 0AN.0 hm~.ol 00N.0l ~0a.0I 0ma.0l aoEEd . 0 #hm.0 m~¢oo mcnoo omNoo hm0.o nnaoo 000.0 NNN.O m¢~ool o¢~.0l O«NoOI oMN.OI vaxrm mjuuH Lw m.Amm mo .02 0 n N a q n N H c n N H Ilewmmmummmmmm uUOfipsm mo Ho>oA H. N. n. «co m 00 33330: .Hdaz on: mucouuu coauuauouaH 0:1 Aaaalaoz on. auuuuuu ouauuux vouuomox .voooouo «swan .auun u9030uo:0H0 unu no vegan aoHuoauou:H macaw an nuuonnsm you nouasvm ecu: ecu can auuonaam you concavm sou: any aouauun occauaauuuoo any .um canoe mm0.0 ~0m.o m0¢.3 mam.0 000.0 00N.o o;~.0 00H.0 n0H.0 :oH.o 000.00 NN0.0! aoEEd 0H :Nn.o 0¢V.0 HNN.O 00H.0 o¢~.0 00H.0 moH.o 000.0 0H0.0| m~0.0 Nmo.ol «No.0I voxrm H00.0 001.0 000.3 00¢.0 m~:.0 :an.0 m0~.0 00H.0 00H.0 H00.0 H00.0 0H0.0 aoE:£ m Hn:.0 OH:.0 nmm.0 JON.0 ~0~.0 ~0~.0 HNN.O N 0.0 000.0 0N0.0I 000.0 HH0.0 voxflm mum.3 ~n¢.0 me.0 mow.0 0m~.0 00H.0 HeH.0 HOH.0 0m0.0 HN0.0 000.0! 000.0! aoEEd 0 wo¢.0 000.0 ¢0n.0 Nom.o mnw.0 ¢0~.0 omN.0 mHH.o NN0.0 mp0.0 000.0 HO0.0 nome mo>.o m~0.0 mmm.3 033.0 :mm.0 03m.0 mu~.0 0mH.o NHH.0 0H0.0 mNo.o Hm0.0I Bogafl 0 suing. mnmfiq omm.o 0~m.0 0m:.o HNm.o NHm.0 0010 000.0 000.0: mmo.0l 020.0 ”HHHM» 905m 00 .oz 0 n N H 0 n N 0 uHmco ouuuwm you saw wo Ho>wA nNNH. 000N. mN0n. 0:0 0 mo huaaancnou .Haaz oua uuowwum coHuuuuou:H 0:: Hascucoc oua uuuoumm onu no venom :oHuuuuouaH macaw x0 ouuofinsm new mouoacm use: 0:0 0:0 macaw New mo wusomuz vuumwmom .00000u0 osouu .uuma macaOUOnoua unavm can: an» cooauom occauaaouuoo onH JLM mange APPEND IX C The listing of the computer program employed in the Monte Carlo procedure. 132 Nna aux \\ zouk(4uazou mo ozw z o zozzou . mo. mhzmzmmHaowm wmou mUOH owhmoaanm mmmak~mo >Iu J~<>< km~mo 004 h 000 \\ A woo..ma..o..m..p..o..m..¢..m..N..H.\HNH.HuH.HMwhao «“40 zmo . mzqmz.N>zz.H>.o>. .HNN.HN.ON. .rzHoa.Fa0 .HHI.IV MUZmJ<>Haom zmo HNHszzz ..o.mVOIm .Hou>>> .HoHovxxx zmo . .Na.¢.m0N> .HNN.NHOH> .H¢0fl00> zmo HNH.¢.m0NN .HNH.NHVHN .H¢¢HO0N Zuo HNH.NHONI .HNM.¢.n0I Zmo zwuoma ow» 0m» 005 omb 0N» 0H» 00% 000 0m0 0N0 000 omo 0¢0 0M0 0N0 0H0 000 00m 00m 05m 00m Zwo zwo Zmo zwo zwo zmo zmo zmo zwo zwo Zmo 2m0 zmu zmo zmw wnzmhzou .NH.HIW.H0.H004~»20. Hmh.N0. mFHmz oH.HaH w on szHFZO0 0 OF 00 HNN.HI1.40.H4m4~k200 Hmw.o04 o m,nflovuo Hmzmmh._ n:mvuo thmtmfrkmzrwnaxvmo m210n400uo “\LIWTX antvéQf <1.U../_mu" “~Im#miavla¢0uo ,mvuo r}mu.mvmo mxanflmvmo mzmurH,uo Fokzu.0pzm Nmz-.mzm Nwzukmzm mmznamzm Nz.m :m2 .0»: kz¥c zuir.r ” “z.mzuhnz mztmzammz I." 135 om¢a 0¢¢H om¢fi 0N¢H 0H¢H 00¢” 0004 omma ormu 00ma 0mma o¢mfl omnH ommfl oflmfl 000d coma ommH ONN“ 00~H 0mNH 0¢NH OMNH ONNH OHN” 00NH ooflfi 04H“ ONHM 004M omaa 044a OmHH ONHH OHH“ ,ooflfl 000a 000a OhoH .000fl _omOH .0904 .0m0H _owofl .900“ ..—_ '— \.. .‘Pot ...... ..... .... N4wuflfl.m+nvma00 H4wnfio+0.o+vam00 H4wu40+0.0+wvam00 H4wme+fi.n+Mvam00 fl4munfi.wvmm00 m.Hu0 0M 00 m.HuH 0H 00 ¢4m.n4w.NHu.H4m .moH.oo. aqua .mz.Hua.HnHN>zz>> vxxx H co ..HuH H on .ouHH.wrhzsco . _ ._ _ NH.HuH HH on oH.Hnw 3H 00 ourn.H0hzaooH co on O a V“ II U u\ [p H he ('0 4 Q 1 I ) N 4 Co 00 002 N (\J ‘4 0" (‘ ) l" r‘iv r41- connfi.H4» . Nd.Hufi a . .. N.fluH N 00 .. . MDZHFZOU . .TUmm noboflov whHMR “Ummvozouwmuumw '1. P‘ I" u»... w..... .. . H ._.:.0NH.Hrw.4T.H0mHH»zu..H¢N.Hoo whHmz oH.HuH 9.00 hmF¢HOV.MkH&3, H»>.H00ithma H.. H ......,._ 0xm. ma ha 0H In '4 ¢H x“ r ‘ 136 zwo zwo 2mm 2mm zww zwo zww mz.H.w nN oo Pz.HuH mm on HwHH>zuH¥.w.H.N> onz.Hu¥ 0N oo mz.Huw ¢N oo Fz.HuH am on HnomznHw.H0H> onHw.HVH> Hmz.HuH mm on mz.Hun NN oo Hw.u.H~.Hx.H0H4oxu+Hn.H0H>an.H0H> sz.Huy NN oo .onHw.HHH> pmz.HnH NN oo mz.Huw NN oo .Hlvzz mo mHm>4uHHO0N hohz.fluw mm 00 ¢m O» 00 HH.Om.mZthv mH ZHPZumEHHz N.HquHz 0m 00 MDZHPZO0 .ouHH.fi.y0NN mm OF 00 .HuHH.fi.xVNN om.am.dm H.0lfiu.fi.xvm>0 m” 4H.0.val.u.0.¥0N>nHN.0.xvm> mz.HuH mm 00 mzaflufi mm 00 wz.aux mm on >ZO»0~0 IUHI3 h< w4HkZz mzamonzux mm 00 ‘- C Hm 0m mm mm hm 0N mN UUU K) - —- -- _..H -— -41.“. -_—... -. 4.... _“q— -- _.._- H. _ .. -H.__.- ...—H fl—-H§m-mHfl—.M.———.m~ ...—..._ 138 oomw oosw omhw chww 00- enha o¢- onhw ouhm OHH“ oosN owo~ Onom oN0~ ooom om0~ o¢0~ on0~ omoN oH0~ 000m comm comw ohmw can” Omnw oqn~ 0no~ oun~ onN oonm oo¢~ 004~ chew oo¢~ 0n¢~ 0¢4~ anew o~¢~ oH¢~ oo¢~ ooMN .ommN osnm .ooNN 200 200 200 Zmo 200 200 200 zwo 200 (4.0 700 Zwo 200 ZwO zmu Zmo 200 200 200 Zmo 200 200 zmo 200 rmo 200 200 200 700 200 200 200 200 200 Zwo zwo zmo 200 200 200 200 200 200 zwu " -~ -.~..~... n- 4 Ha.pmzam..1.»mzam+oomuoom «z.H.u on as HHHNxzaw.HH.Nmzam+m..Hovkuum Hammumamu.m.Humm H«HNuuus..mzm.zzm.u.mmzm.>a...Hpvkumm .NOHummu.a..;cu.mc...o_»uuw .H.bumuu.mmmmnmma...m0Fuuu .H.puum:..mmm:»am.u.¢0»uum 44m . aoa+omncm 4 a+.x.0.mzawuHx.fivszm n m+.¥.~.m:0mu4¥.~0m20m n+.0.~0h20mu.fi.H4»I3w a+.¥.mwybmu.gvmmyaw Q+Hfivh0200u400meaw a+H~vmeDmu4~vmeDw a+44oonou)a 0 mo m..Har>z 0 WC ..u..s.__.~...(. w 00 .o o .J a I ‘ .w53?t~.?0.¢.‘.t_\ ...".H\r.ua \........ t I: ‘1 .0 . a ' J H‘\ ‘ “0v-‘.>~ o 1’ Abs.oorx;04\ HF3c.v:nV\ uauxxa :H.¢Hu:a ‘ ’0‘ p N lac \|\.\.C 3).: L....cc..:c I) ,.l‘ 9) r, ’1’) '4 rotq'c .....n H fl‘" "N” ""“" " 000’) 0’10") ‘2’)000000 k. ... D :1 \ \lv ’) mo U: “I L) mamnr‘nnrsnfino flfl\flfif\f.r.ofi "J C: ’) XXIII/{IIIIII ... "O . ml:\’) ’) ’1 O O r) "J O") ’) f. ’. 9 fdfo ’o" P»: ’0‘: N 'a H 'o (J H '» H '4 HI. '4 m r. «‘3 n r‘ «A 7///(Z//7 L...LHL~' '54 J. «a z..:..\ 1 at .... :9 H0 . szrxr0v.:Cc an no Nzcxx Luzmtu 0m00 4440 Hu.fi.m0NN+Hx.fi.N0NN+H1.0.HVNMnHVLfivcxx mz.Hay mm 00 . mZLHnfi mm 00 n+.fl.~vhzaouuuHfl.H0h20004 HHHvszoa.moaHfivp0mm. m mH.Hn0 mm 00 0H.~uH wm 00 HNuzm+AH0>>>nHH0>>> 0.4"” Hm 00 .fivxthuvzm+Hfi.Hvxxanfi.H0 xx 0.4ufi on 00 0.4um 0m 00 «+.~.00»23001.~.00k2300 HHH.0VM4HH20.moonvu0 mm Nuoanu o¢ 00 0M.Hufi 00 00 H0a.fi0x nHmHVFUmm H>.H0xmnhuflvk0mm H¢.HVanHHm0»0mm 4N.HvxmnHOHvF0Lm 00 on em Ln ~O m «0 L0 x? «Q LG (‘1 m am on 143 omom omom OHOm 000m 0000 000¢ o>0¢ 0000 0mm¢ 0¢0¢ 0m0¢ 0N0: 0Hm¢ 0000 000¢ owm¢ Ohm: 0000 0000 0000 0m0¢ 0Nm¢ 0H0¢ 000¢ oahc 0000 0hh¢ 00>¢ 0mw¢ 04h¢ 0mh¢ ombc 0Hh¢ oowc 000¢ 0000 Choc 000¢ 0m0¢ o¢0¢ 0m0¢ 0N0¢ OH0¢ 000¢ 00m¢ Zwo zw0 Zmo 2m0 zwo zmo Zmo Zmo zmo Zmo ZMO Zmo Zmo 2mm Zmo Zmo zmo Zmo 200 Zuo zmo zmo zmo zmu Zmo 200 200 zww 2mm ZMO ZmO Zm0 Zmo Zwu ZmO Zmo Zmo zwo Zmo zmo zmo Zwo Zwo zwo sz LLHo.zL Lam.Hov mpHmz LLHLa<>LLmomnHHLm<> ooHuH 00 ca Ho.HuH..H0»>>0 HNm.HoL wkas Lo.HuH.HH040 Hom.Hoe mLHmz merHoL MLH 3 fimb.H00 thaB Hmz.Haw.Ha.N>zz«mzv L¢m.HoL whHm: . HmN.HoL whHms mz.monz.onz HmogHoL meHaz Hm».Hov mpHma LLHo.z>>nHHV>>> o.HuH mo 00 mm<>\hfi.HvOImuH0.HLOIm amm<>0hmomumm<> .fivm<>aHHOm<>nmm< HH00>>>8H00>>>01400m<>n400aL) 40.00xxx¢z HHH0>>>tHH0>>>VIHH0m<>uHvaq> HM.H4xxxkz Hn00>>>*HHO>>>0lnfi.mVormufifieHVOIK Hfl.HVXXX$Z0 00 ¢0 m0 N0 H0 144 omcm 0h¢m oo¢m omcm o¢¢m 0m¢m 0N¢m oH¢m OOdm 00mm 0mmm 05mm 00m; OmMm 0¢mm ommm ommm onm 00mm Comm Ommm osmm comm ommm oqwm ONNm Ommm OHNm Comm 09am omflm cram ooflm Oman oqflm Omnm ONHm .14 ..r ooam 000m 000m _owom ooom anon 020m zwo zwo zww zmo zwo zmw Zmo zwo zmo zmo zmo zmo Zmo ZmO zwo zmo zmo zmo 2m. zmw zmo zmo Zwu 7w0 zwo zwmq zmo nmz.flnu.nnq~>zz<20Hm.bmuz.zzzL “NH.Hufi.nfi.MVPZDOUu« m:mhzw Andoflnfio.fi.avwzaouv.AHVMPEL .mz.fluuo.~v~>zz UFHEZ ukmmz JL'J; Fl— UJ UJ LU U1 UJ J 'J H 05 r—o e—aH {ICC .. 2: ‘3; E: ( AL. a: Q .- (if m .2. m .u—i. m .2. a; r. 7.: 1.3 (... m3 a3 ( C3 :- r— r« r— r— p— r— r— r— r- ?— r. F~ }— r- r— m3 Z3 ~.u. 3 m._2_ ml< «3 E. « F— }- :—- }— r- :- r— z~ f'. 5‘. H H H [H4 .-4 |-C F-l H F. ... F‘. f. >4 MO h-Q hi H H O-C wthB IL... . 145 zmo zwo zwm zmo zmo zm zmo zmo 2mm 2mm Eu .3 zuo zmo zmo Eu zmo zmo zmo zw zwo .8 $0 $0 23 zuu zwo zwo .mo zmo zwo Zmo zmo zmo zwo zmw zwo zmo zwo zmo zwo zwo zuu “\xw hqzmom Aarav Fqkmom .0a.o~m.xo¢\\\. Fzz.flo. wwwmx amm.Hoy mpm‘: 24.nufi..fi,~voxmv A¢ofloaov mFHmS q.HuH oo oo .Amofi.flov whuaz mam ”mofl.flov mhmmz .o~.~a~..m~.~v»zoouuv.«mv¢»2u “ooH.Hcv muum3 no~.Hu~..NH.HVPZDOUHV.flb_¢w1u Aoofi.flov mkmmz fio~.~u~..HH.HVPZDOUHV.Rovcpim Aoofloflou whmmk flow.fluu.noH.Hvkznoqu.Amv¢hzm “ooHaHnu mpnmx “oo.flo. thmz ”Mofl.fiou whmmx “mm.acv mpmax .mz.~u~..~.~>zz thI¢moxo¢\v +Izm0m zwo “\omonPIJmmmOU MI» mo mz.0H»Dm~me0 JIUHmHaSuIN¢.x@mV §,<0L zwo A\\mHON. QHI.XmV IFOOL 2mm ~\.0oa o. 0. so 0. m. ¢o no N. a. 0 Immofl 2mo moi No! «.1 Io! mol 0ol 5.! mo! 0.! 0.-IIam.XmHv (£20m zmo A\.mhuwmmw MI» mo mzokamMMBmHo JIUHmHazm mmhxmc.xmmv w<3oom 2mm “\Axm.mH.XavNHo0HI\V FISoOm Zwo ~\.0mm30wo oohmrofloxIHV FIzmom Zmo h\\AXa.m.0uvm. mZImz Iofloxmflu FIEMOL zmo A\\OH<0¢KmNV FIFK,1 zmu h\. mmd ZmOKIDOm ZIwz UI.F m0 mZOHkIH>wo 0mIQZIkm OZI mZIm: MIPIQmQXmmv HIHmou zwu “AXN.m.mm.:NV¢. x c.hxm.m.mm¢XNv¢axov I>mOL Zwo Am nH.IDDOmIk I0. m“ mmmzaz mhvmfioaw wow 20” PINIzohoumotm Zwu¢m.xmm.NH. IODOmIP H mmmZDZ WFUmmem Lou ZOHHINI>O OUHQ_m¢.xH-\.OHN zwo hzmomuwwz< mmmOuw zPJDUHmmH0 ION.anOHhIH>mo Iflflvmo Qwhmwz N zuo mmJQEIw ZMkH owkJJ :xux: zmo Iuwoa omm>4wxuzofiom Howaoa JHaQI owmn>mx azzmtf "\ O 0 O B O O>< MIL u7m ’ (— .Omoa.am kmDDDI .aQIUomoz oz< >IEOJJOI xMIJU .mema.xm.zzazzv hmmom mzuhsozmam a“ In (Q P! &)U 148 oooH 000 000 050 000 0m© o¢o 0mm 0N0 0H0 000 can own Ohm 00m 0mm 0¢m 0mm 0mm on com com own on» m». oqh ONQ 0H0 000 I I C 09m. 00m. Ohm . 00m. mHo mHo wHo mHo mHo mHQ WHO wwo mmo wHQ mHo mHo mHo. mHo. mHo mac A Nm.mm.Hm In.0m.om H3flu wc. h. Cupd _Q..1 zc~_:c»aymfic ...” —-w‘q‘”o. \ofiC (AC. 0 (O ,\.mka:c,\.,)_o . .1.—..\>. — _ I. :HHZ .zx\zx.OOJIt.oom\Xxu+ ¢N.m¢¢mm C fly up» ..CN. . 02H_;§I.:Q OOONRDU Oom\(»4monm OQN\H ZX+2K H$> Oo N\HOoN IEXVHN> _:UItu H A~.ou v m OOH”: _ Ui. !; he >- HHUIuVOOJI+JHUILu.0UIu mmoommom OOOtFUImusUI_ I .‘. a. t A; .. omom mJNNN. H\.UOU >OU >OU >OU >OU >OU >OU >OU >OU >OU >OU >OU >OU >OU >OU >OU >OU >OU >0 >OU >OU >OU >OU >OU 02m ZMkam szthou .fi. my xnuAHofivxm “«Afi.figxx*am. Hvxxvhmom.\flfi Hvx XKHéVGHVXu QZIJufi m 00 aonuH m on ooHlmo.Hu~fi.Huxx no. 0 Owoafiovaxu I . OZIH a on H :_/_ b< j AfioHUXvu m«\n.fivm>$flmvm>lflfi.vix yomv>+flfiomvxwa «éxuuu ocuhw. QZQ- << ..4 b .253“ d \. o V, (13 *4 .N ha o—c K II) >< : lOXO -Cx>0-1© .. I ‘ 5. az.Hu ax.“ v>+hm um> u“ xx an 5’ "'730f0 .\ \o O O (I ,_ QC c-l l-- r( *4 0‘ ..«z a>zo>zvxm on>zo>zvxx ..oova> onwz.>zu> onwzmr szo>zaxm.xx.mzouz.>~ mI>OU mszaomm I CW 152 00¢ om¢ o¢¢ om¢ om¢ oH¢ 00¢ oom 0mm owm oom 0mm o¢m 0mm om odm 00m com 0mm Ohm oom 0mm o¢N onm 0mm OHN oom 00H Oofl ova on” omfl o¢a om” ONH cafl man man man max man man mma man man mac mam man man man man mma mma mag maa mma mma mma mma man man man mac mam wan man man mma mma Qua mma ozw zmabwm.» mxfiDH meWt.am.4:<0|.<.J2uhirnw. LN .H+Fom-+oa o H+QO§HQ_0.—. XMUP+xanuzn UX*A.OM\QO_ V3? Musufmm R IDnn lxxm.~ oHH_Hth o HSFDW o a+®flb0® .«nuOh ¢¢0 00¢ mmlmX< umm i ZOHWZXCU or:-zow oflnzou ... l.. .c.¢.l\l|( x U< V’NOC... U} aim Pu m H~’Q Z‘i (DU 1 ‘1 \‘f O Q 0 m c!" O o H + M. II S-~-’._COH“( —03 «I (1 ll azm ~. 0 U :0 O U) U- U): 4 u. (D K «L U u) H-r— «. r—‘M "a \ O (I LC ~.fl.m azmp\u$$munm 3.1.0 0 nu .m.>.3~momom zomkuz;u I ... D 7"! 153 00¢ 0n¢ 00¢ omq 0q¢ 0m¢ 0N¢ 0H¢ 00¢ 00m 00m 0pm 00m 0mm 0¢m 0mm 0mm 0am 00m 00m mm ohm 00m 0mm 0¢N 0mm 0mm 0am 00m 00” 00a 0rd 00H Oma 0¢H om“ oma OAH Zh¢flmm¢hmflooom mmmflmfl¢hbvflmm¢bwflooom kmz<: O OOON= > Zeird ozm