ABSTRACT AN INVESTIGATION OF THE EFFECTS OF SELECTED METHODS OF POST HOC BLOCK FORMATION ON THE ANALYSIS OF VARIANCE BY David John Wright Post hoc blocking refers to the imposition of a gene- ralized randomized blocks design on observations from a simple, one-factor design with fixed treatments when additional observations are available on some concomi- tant variable. The three post hoc blocking methods were: (a) post hoc blocking within treatment groups (PHB/T): within each treatment group, the experimental units are ranked on the concomitant variable and divided into b blocks of equal size; (b) fixed-range post hoc blocking (FRPB): b blocks of equal size are formed in the concomitant variable's population distribution and experi- mental units are assigned to blocks within treat- ment groups according to the b-l population quantiles of the concomitant variable; (c) sampled-range post hoc blocking (SRPB): the experimental units in all treatment groups are pooled, ordered on the concomitant variable, David John Wright and then assigned to blocks within treatment groups according to the b-l pooled-sample quan- tiles of the concomitant variable. For purposes of comparability to earlier research, attention was restricted to the case of dependent and concomitant variables with homogeneous bivariate normal distributions across treatment populations. For each post hoc blocking method, the major topics were: test- ing the hypothesis of null treatment effects under con- ditions of block-treatment additivity; testing the hypo- thesis of block-treatment additivity; and the precision of the method in the additive condition, relative to the one-factor analyses of variance and covariance. The investigation relied heavily on Monte Carlo methods to generate empirical distributions of mean squares and F ratios where analytic solutions were either unknown or unreasonably difficult. For each combination of treat- ments, blocks, average treatment-block cell frequency, and population correlation (p) between the dependent and concomitant variables, 1000 samples of observations were generated. Within each sample, equal numbers of observations were randomly assigned to treatments and then assigned to blocks within treatments according to the three post hoc blocking methods. A one-factor analy- sis of variance was computed on the unblocked observations, David John Wright followed by unweighted means analyses for the three post hoc blocking methods. If any of the methods had zero cell frequencies, the sample was dropped for that method and additional samples were later generated to bring the total number of samples to 1000. The PHB/T method was of no value in testing the hypo- thesis of null treatment effects in the presence of either block or interaction effects and provided no gain in precision when both were zero. Treatment, block, and interaction effects were comfounded in the test of the null treatments hypothesis. The PHB/T method is excluded from further remarks about the post hoc blocking methods. Blocks, in the FRPB method, were a fixed factor and the ratio of treatments over residual mean squares followed the central F distribution under the null treatments hypothesis, regardless of the degree of block-treatment non-additivity. The FRPB method had greater precision than the one-factor design only for p 3 0.6. Unless block-treatment cell sizes happen to be very nearly equal, the FRPB method does not offer appreciable gains in pre- cision. A priori blocking or the analysis of covariance, when applicable, are typically more powerful. The FRPB ratio of interaction over residual mean squares did, however, provide a means for investigating block-treat- ment interaction or, equivalently, regression heterogeneity David John Wright in the analysis of covariance. For the specific cases generated, the test of the block-treatment additivity hypothesis showed reasonable power, particularly for samples with nearly equal cell sizes. Blocks, in the SRPB method, behaved like a finite factor. In the presence of increasing block-treatment non-additivity, the ratio of treatments over residual mean squares was increasingly liberal, while the ratio of treatments over interaction mean squares was even more conservative than the former ratio was liberal. The same conclusions appeared to be valid for the a priori sampled range method where equal-sized blocks are formed prior to assignment of experimental units to treatment groups within blocks. Neither sampled—range method pro- vides a clear test of the treatments null hypothesis unless sample sizes are very large, little or no interaction is present, or the test is conditioned on a prior test for non-additivity. For the SRPB method, the test for addi— tivity had reasonable power when the interaction was great enough to make the test for treatment effects markedly liberal. The SRPB method, like the FRPB method had greater precision than the one—factor analysis of variance for p 3 0.6. For purposes of increasing precision, both a David John Wright priori blocking and the analysis of covariance are pre- ferable. Unless cell sizes are very nearly equal, even the one-factor analysis of variance is generally more precise. Investigation of block-treatment non-additi- vity on an after the fact basis is the only clear asset of the two post hoc blocking methods. AN INVESTIGATION OF THE EFFECTS OF SELECTED METHODS OF POST HOC BLOCK FORMATION ON THE ANALYSIS OF VARIANCE BY David John Wright A THESIS Submitted to Michigan State University in partial fulfillment of the requirements for the degree of DOCTOR OF PHILOSOPHY Department of Educational Psychology 1971 To my grandfather, Harley 2 Wooden ii ACKNOWLEDGEMENTS I would like to thank the members of my doctoral committee, Professors Andrew C. Porter, Robert C. Craig, Maryellen McSweeney, Judith Henderson, and John Vinsonhaler for their encouragement and assistance throughout my gra— duate program. Particular thanks are due to Andrew Porter, who has been a stimulating teacher, an uncommonly atten- tive advisor, and a good friend. I would also like to acknowledge the contributions to my graduate studies of Professor Charles F. Wrigley and my colleagues Howard S. Teitlebaum and John F. Draper. Charlotte Hayes pro— vided patient and accurate assistance with the typing and proofing of the final draft. Above all, I am grate- ful for the understanding and support of my wife, Carole, who still can't believe it is finally finished. iii TABLE OF CONTENTS LIST OF TABLES LIST OF FIGURES CHAPTER I Introduction Post Hoc Blocking Methods Scope of the Investigation CHAPTER II. Description of the Investigation The Normal Theory Model The Generalized Randomized Blocks Model Precision A Generalized Index of Apparent Imprecision: I Precision of the One- and Two-Factor Designs Within Block Variation of the Concomitant Variable Data Generation Random Number Generator Generation of Dependent Variables Block Formation and Analysis Design Parameters Used in the Investigation The Number of Treatments The Number of Blocks The Average Number of Units Per Cell The Correlation Between Dependent and Covariables The Non-Centrality Parameter Additive and Non-Additive Conditions Empirical Cases Generated iv vi ix l4 l6 18 24 27 29 32 36 36 37 40 41 42 42 43 44 44 45 46 CHAPTER III. Results of the Investigation The Case Against Post Hoc Blocking Within Treat- ments The Hypothesis of Null Treatment Effects Empirical Average Mean Squares Empirical F Distributions Block-Treatment Non-Additivity Precision Comparison to A Priori Block Formation CHAPTER IV. Summary and Conclusions Post Hoc Blocking Within Treatments Fixed-Range Post Hoc Blocking (FRPB) Sampled-Range Post Hoc Blocking (SRPB) BIBLIOGRAPHY APPENDIX A APPENDIX B 49 54 55 66 78 82 88 93 94 95 98 101 102 LIST OF TABLES Expected Values of Mean Squares in the One- Factor, Fixed Effects Analysis of Variance. Expected Values of Mean Squares in the Gene- ralized Randomized Blocks Design with Fixed Treatment Effects and Fixed, Finite, or Ran- dom Blocks. Experimental Units are Random. Expected Values of Mean Squares for the Post Hoc Blocking Within Treatments Method Empirical Average Mean Squares for the One- Factor Design and the Post Hoc Blocking Within Treatments Method: Additive Case, ¢ = 0.0, t = 4, b = 4, n = 5, p = 0.0 - 0.8. Observed Type-I Error Rates for the One- Factor Design and the Post Hoc Blocking Within Treatments Method: Additive Case, ¢ = 0.0, t = 4, b = 4, n = 5, p = 0.0 - 0.8. Empirical, Mean Squares for the One-Factor Design and the Fixed- and Sampled-Range Post Hoc Blocking Methods, Additive Condition: t = 2, 4; b = 2; n = 5; p = 0.0, 0.6. Empirical Mean Squares for the One-Factor Design and the Fixed- and Sampled-Range Post Hoc Blocking Methods, Additive Condition: t = 2; b = 4; n = 5; p = 0.0 - 0.8. Empirical Mean Squares for the One-Factor Design and Fixed- and Sampled—Range Post Hoc Blocking Methods, Additive Condition: t = 4; b = 4; n = 5; p = 0.0 - 0.8. Empirical Mean Squares for the One-Factor Design and the Fixed- and Sampled-Range Post Hoc Blocking Methods, Additive Condition: t = 4; b = 4; n = 3, 10; p = 0.0, 0.6. vi Page 17 22 50 52 53 56 S7 58 59 Table Page 10. Empirical Mean Squares Under Conditions of Non-Additivity for the One-Factor Design and the Fixed- and Sampled-Range Post Hoc Block- ing Methods: t = 4; b = 2; n = S; p = 0.6; ¢ = 0.0. 61 ll. Empirical Mean Squares Under Conditions of Non—Additivity for the One-Factor Design and the Fixed- and Sampled-Range Post Hoc Block- ing Methods: t = 2; b = 4; n = 5; p = 0.2 - 0.8; ¢ = 0.0. 62 12. Empirical Mean Squares Under Conditions of Non-Additivity for the One-Factor Design and the Fixed- and Sampled-Range Post Hoc Block- ing Methods: t = 4; b = 4; n = 5; p = 0.2 — 0.8; ¢ = 0.0 63 13. Table 13. Medians and Ranges of the Ratios of Average Mean Squares in the Additive and Non- Additive Conditions: One-Factor Design and the Fixed- and Sampled-Range Post Hoc Block- ing Methods, ¢ = 0.0. 65 14, Observed Type-I Error Rates for the Ratio MST/MSR: One-Factor Design, Additive Con- dition, ¢ = 0.0. 68 15, Observed Type-I Error Rates for the Ratio MST/MSR: Fixed-Range Post Hoc Blocking Method, Additive Condition, ¢ = 0.0. 69 16. Observed Type-I Error Rates for the Ratio MST/MSR: Sampled-Range Post Hoc Blocking Method, Additive Condition, ¢ = 0.0. 7O 1?. Observed Type-I Error Rates for the Ratio MST/MSTB: Fixed-Range Post Hoc Blocking Method, Additive Condition, ¢ = 0.0. 72 18. Observed Type-I Error Rates for the Ratio MST/MSTB: Sampled-Range Post Hoc Blocking Method, Additive Condition, ¢ = 0.0. 73 vii Table 19. 20. 21. 22. 23. 24. 25. A1. B1. B2. Observed Type-I Error Rates for the Ratio MST/MSR: Fixed- and Sampled-Range Post Hoc Blocking Methods, Non-Additive Condition, ¢ = 0.0. Observed Type-I Error Rates for the Ratio MST/MSTB: Fixed- and Sampled-Range Post Hoc Blocking Methods, Non-Additive Condition, ¢ = 0.0. Observed Type-I Error Rates for the Ratio MSTB/MSR: Fixed- and Sampled-Range Post Hoc Blocking Methods, Additive Condition, ¢ = 0.0. Observed Power of the Ratio MSTB/MSR: Fixed- and Sampled-Range Post Hoc Blocking Methods, Non-Additive Condition, ¢ = 0.0. Observed Power of the Ratio MST/MSR and Empiri- cal Indices of Imprecision: Additive Condition; ¢ = 1.0; p = 0.0, 0.6. Observed Power of the Ratio MST/MSR and Empiri- cal Indices of Imprecision: Additive Case, ¢ = 1.0, t = 4, b = 4, n = 5. Generalized Indices of Apparent Imprecision for the Fixed- and Sampled-Range Methods when Used in the Design (A Priori) or Incorporated After Data Collection (Post Hoc), and the One- Factor Analysis of Variance and Covariance. Summary Statistics for Ten Samples of One Thousand Pseudorandom Normal Deviates Generated by Subroutine RANSS. Empirical Mean Squares for the Sampled-Range A Priori Blocking Method with Both Additive and Non-Additive Block and Treatment Effects: p = 0.6; t = 2; b = 4; n = 5. Observed Type-I Error Rates for the Sampled- Range A Priori Blocking Method with both Addi- tive and Non-Additive Block and Treatment Effects: p = 0.6; t = 2; b = 4; n = 5. viii Page 75 77 80 81 85 87 89 101 102 103 LIST OF FIGURES The Generalized Randomized Blocks Design Three Post Hoc Blocking Methods with Four Blocks Each Imposed on a One-Factor Design with Three Treatment Groups and 4n Experi- mental Units per Treatment Group Regression lines the the Additve and Non-Addi- tive Conditions: t = 2, b = 4 Summary of Cases Investigated Empirically ix Page 10 39 47 CHAPTER I INTRODUCTION Researchers are often interested in testing an hypo- thesis that t treatments or levels of an independent variable have had an identical, additive effect on some population as reflected in a dependent variable. A com- mon and straightforward way to test this hypothesis is to draw t random samples from the population of interest (or randomly assign elements of a random sample to each of t subsamples) and subject the samples to the treat- ments. If, in this completely randomized experiment, treatment effects are additive in the metric of the depen- dent variable, experimental units operate independently, and the dependent variable is normally distributed with equal variances across treatments, then the one-factor analysis of variance F statistic provides a test of the experimenter's hypothesis. Given a basic research paradigm of randomly equi— valent treatment groups, independence, additivity, and a dependent variable with homogeneous normal distributions, much of experimental design is concerned with increasing the precision of the experiment by minimizing the stan- dard errors of the treatment means. Cox (1958, pp. 8-9) 1 2 considers precision as a function of (a) the intrinsic variability of experimental units and the accuracy of the experimentl; (b) the number of experimental units; (c) the experimental design and method of analysis. The first of these factors, intrinsic variability and experimental accuracy, is generally least amenable to manipulation without affecting the experiment's exter— nal validity. For example, a restricted range of experi— mental units might be used to decrease intrinsic varia- bility or materials might be presented by videotape to increase the accuracy of presentation but then the experi- ment may no longer generalize to a larger population with live presentation of materials. Experimental pre- cision is approximately proportional to the number of experimental units per treatment group; increasing the number of units is a direct but often prohibitively expen- sive way to reach a particular level of precision. Pre- cision can sometimes be increased appreciably if addi- tional information on concomitant variables, available prior to the experiment, is incorporated into the experi- mental design and analysis (e.g., formation of homoge- neous blocks of experimental units) or into the analysis alone (e.g., analysis of covariance). This increase Intrinsic variability and experimental accuracy refer to unit and technical errors, respectively, in the terminology of Wilk and Kempthorne (1955, 1956a, 1956b) and Scheffe' (1959). 3 in precision can be by a factor as great as one minus the squared correlation between the dependent and con— comitant variables. While a number of ways exist for utilizing infor- mation on concomitant variables in experimental design and analysis, three methods are of interest here: (a) analysis of covariance; (b) generalized randomized block designs: (c) post hoc blocking, the major topic of this paper. The analysis of covariance statistically removes variation associated with one or more concomitant variables and is quite thoroughly covered by Cochran (1957) in the lead article for an issue of Biometrics devoted entirely to that topic.2 Some of the analysis of covariance's principal disadvantages involve stringent assumptions about the relationship between dependent and concomitant variables, i.e. that the relationship is adequately speci- fied (linear, quadratic, exponential, etc.), that the regression is constant across treatment pOpulations (Peckham, 1969), and that components of the model are additive. A further assumption about error free measurement of the covariate is of little consequence when groups are randomly equivalent (Lord, 1963; Porter, 1967, 1971; DeGracie, 1969). 2See Elashoff (1969) for a more recent review and summary of research. wv—u ”H Blocks or levels designs refer to the class of experi- mental designs where blocks are formed by homogeneously grouping experimental units according to some antecedent characteristic, then randomly assigning units to treat- ment groups within blocks. Typically, blocks are formed or sampling is carried out in such a manner that an equal number of experimental units are assigned to each block and treatment group combination. No distinction is made here between blocks, levels, and strata according to the concomitant variable's type of measurement scale as did Glass and Stanley (1970, pp. 493-494), for example. Also, while the blocks terminology has sometimes been reserved for the case of a single experimental unit per block-treatment combination, Wilk's (1955) "generalized randomized blocks design" terminology will be followed with nij experimental units per block-treatment combi- nation, depicted in Figure 1. Yijk denotes the value of the dependent variable associated with experimental unit k in the ith block and jth treatment combination where i = l, 2,...,b; j = l, 2,...,t; and k = l, 2,..., nij = n, a constant, in most cases. Cox (1957) and Feldt (1958) investigated the preci- sion of designs utilizing concomitant variables, includ- ing analysis of covariance and two forms of blocking. They agreed that blocking tended to be more precise for low correlations between dependent and concomitant variables while the analysis of covariance was more precise for Treatments Block 1 3 t Ylll Yljl Yltl 1 Yllk Yljk Yltk Yiinl1 Yljnlj Yltnlt Yill Yijl Yitl i Yilk Yijk Yitk Yilni1 ijnij Yitnit Ybll ijl thl b Yblk ijk thk Ybinm Ybinbj Ybénbt The Generalized Randomized Blocks Design Figure l very high correlations, but did not agree on the transi- tion point for the relative precisions of the two methods. Pingel (1968), in a Monte Carlo study of generalized ran- domized blocks designs, distinguished four methods of block formation: (a) fixed-value blocks: b different values of the blocking variable are selected and then, for each of the b values, tn experimental units are randomly sam- pled; (b) sampled-value blocks: b values of the block- ing value are randomly sampled and then, for each of the b values, tn experimental units are randomly sampled; (c) fixed-range blocks: specify b mutually exclu- sive ranges of equal probability in the blocking vari- able's population distribution and then, within each of the b ranges, randomly sample tn experimental units; (d) sampled-range blocks: randomly sample btn experimental units, rank them according to their values on the blocking variable, and then randomly assign the first tn units to treatments within block one, the next tn units to treatments within block two, etc. Pingel concluded that design precision was a function of block formation with fixed- and sampled-value methods most precise and the widely used sampled-range method least precise. Post Hoc Blocking Methods Post hoc blocking, a term coined by Porter and McSweeney (1970), refers to a class of blocking methods which impose a generalized randomized blocks design on data collected from a completely randomized experiment. Block formation is accomplished after experimental units have been assigned to treatment groups. The post hoc blocking methods utilize information on a concomitant variable which is antecedent to, or at least independent of, treatment effects. This independence restriction is the same as that imposed on potential covariates in the analysis of covariance. When data are available on some concomitant variable in a completely randomized experiment, a post hoc blocking method might be considered under one of the following conditions: (a) Heterogeneous regression: An analysis of covari- ance was planned but preliminary analyses revealed that the regression slopes for the dependent variable and covariable varied across treatments, implying nonaddi- tivity of treatment effects, or a blocks by treatments interaction if the covariable were used to form blocks. In this case the blocking method is post hoc both in the sense of being applied after units were assigned to treat- ments and also as a method of investigating the violation of the additivity assumption. (b) Nonlinearity: The form of the dependent-con- comitant variable relationship is not linear, but not well enough known to use nonlinear analysis of covariance techniques. Nonlinearity could be discovered in prelimi— nary analyses or from sources external to the experiment. (c) Metric: The concomitant variable is measured in a metric (e.g., nominal) not amenable to covariance adjustment. Examples are sex, some socio-economic status indicators, curriculum groups, or percentile ranks. If the decision to utilize the concomitant variable is made after units have been assigned to treatments, the blocking is post hoc. (d) Precision: If, under some conditions, post hoc block methods yield greater precision than other methods such as covariance adjustment, then the post hoc blocking methods would be preferred on those grounds. For a range of low correlations between dependent variable and covariable, some methods of a priori block formation are more precise than the analysis of covariance (Cox, 1957; Feldt, 1958) and the same results may hold for post hoc blocking methods. Clearly, then, post hoc block formation methods are not only of academic interest, but may also prove to be of practical value for increasing experimental precision or investigation of treatment effect nonadditivity. The three types of post hoc block formation considered in this paper are (a) post hoc blocking within treatment groups (PHB/T); (b) fixed-range post hoc blocking (FRPB); (c) sampled—range post hoc blocking (SRPB). The post hoc blocking within treatment groups method ranks experimental units within each treatment group with respect to a concomitant variable. The n highest units in each treatment group are assigned to block one, the n next highest in each group are assigned to block two, and so on until all experimental units have been assigned to blocks, as depicted in Figure 2A. The dependent vari- able is then analyzed by a two—factor analysis of vari- ance. Porter and McSweeney (1970) investigated post hoc blocking in the context of a Monte Carlo investigation of the relative efficiency of the Kruskal-Wallis and Friedman test statistics, nonparametric analogs of the one-factor and randomized block analyses of variance. The found, for several combinations of treatment groups and sample sizes, that as the correlation between depen- dent and concomitant variables increased, the Friedman analysis of variance of ranks, when performed on data which was post hoc blocked within treatment groups, yielded high Type—I error rates and low power for noncentral 1C) msouo ucmsuuoua non nude: Houcoefluomxw so can mdsouu acmEuaoua mouse nus: cmfimwn noboomnoco a co bomomEH comm axoonm usom cud: moonuwz ocaxoon oom umom manna w musmfih x no cmaooz moamammnpmaoom u on.“ I om..x «\2 M .«c 6 n A. way” .x we moaauumsw moadsmm upwaooa an» 0» mcflnuoouu mucoEummuu awry“: mxooHn ou umcmfimma mum muss: "vogue: mwcmmupwadEmm o and w coaum smo om. u cm. x x u .c z . H u .x alum n\z u :v u fl.: . u n .w . V\/ “Mn ¢\ZICM I .mnC c n A..cvm n mnoum naav .x up wmaflnunsg cos» :w as so vouwvuo. coflum>umunona saw u . . » nmazmom who cu mcfipuoooa mascum ucmEumouu cwruflz .msouu unwauuwuu sumo new mamuoumaom .c mafia mxoomp cu wocupmwm mum mafia: "ponumz mocmmonmxflh no mxuoHa on pwcuwwmu can x no @wuwbuo mum muss: - mnozumz conga: mcoxoofim no: umom macamnumflmsmm can lowed» .m masouo ucmsumwua aways: mcwxuoam no: anon .< no: No“ Ho: Mo: No: Ho: me we aw Cmvk CN¢> CHV> MACQvfi NACQV> HASQVX .v u n u . . . c . . . v .vc ” u n v . qus Hmvs Ans» n1~+cmcs ~Aa+cmcs His+cmvs ms x r uuuuuuuu . u a nu. n unuuuunnon Hmcamh Hasnvh .m: m m 7 m . n .n: m m m m Ham» aAa+cmca omo‘x .lll' llllll -Lrnv'--'l.lil -Iol.-|.l.ll Hm :Hma HAcNV» .m n u u . . . c . . Hmm N .«c u u n N . » axa+c. mN “x Illlrulll|iglnv£|llillIII- IIIU|1 h nu ma AH and» :NH» caps n.a.s ”.c.» H.=.s .a u u n . . . c . . . .— .H: u u . u . d Haas HNHs and» n.~.> ~1H.a Haas» xoodm m N H n N a xoon mmsouo unmeumwua nmsouo ucmsumwua 11 cases, relative to both the Friedman on a priori blocks and the Kruskal-Wallis on simple randomly assigned data. That is, as the correlation between blocking and dependent variables increased, post hoc blocking within treatment groups led increasingly to rejection of the null hypo- thesis whether it was true (central case) or false (non— central case) but the power was low compared to the other methods studied. A preliminary Monte Carlo investigation of post hoc blocking within treatment groups for the one- factor and randomized blocks analyses of variance suggested results similar to those of Porter and McSweeney for the nonparametric methods, leading to this broader investi- gation of post hoc blocking. Fixed-range post hoc blocking, the second form con- sidered here, refers to the case of dividing a concomi— tant variable's population distribution into b ranges such that an equal proportion of the distribution is con— tained in each range (e.g., quartiles, deciles, etc.). Experimental units are assigned to blocks within treat- ment groups on the basis of their score on the concomi— tant or blocking variable and an analysis of variance is computed on the dependent variable. For example, if four blocks are desired, then the concomitant variable's pOpulation quartiles determine the range of scores which belong in each block. Experimental units in the conco- mitant variable's first quartile are assigned to block 12 one, those in the second quartile are assigned to block two, and so on, as shown in Figure 2B. The dependent variable is then analyzed by an unweighted means analysis of variance with four blocks and t treatments. Sampled-range post hoc blocking is similar to the fixed—range method except that the pooled treatment group samples are divided into b blocks of equal size with respect to the concomitant variable rather than splitting the population into b segments or blocks of equal size. That is, the block endpoints are statistics (sample median, quartiles, deciles, etc.) rather than population para- meters. This form of post hoc blocking is similar to the most common type of a priori block formation where all experimental units in a sample are ordered with respect to a concomitant variable and divided into b blocks of size tn each. The tn units in each block are then ran- domly assigned in equal numbers to each of the t treat- ments, whereas in the post hoc method the cell frequen- cies, are in general, unequal. Pingel (1968) found that while this method of a priori block formation is widely used and often presented in introductory texts as the method of block formation, it is also the least precise of the four methods he studied. Other possible methods of block formation, such as equal sized intervals, either fixed or sampled, were deemed unfeasible as post hoc blocking methods for two 13 reasons. First, the blocks would have to be weighted in some fashion in order to obtain unbiased estimates of pOpulation parameters-—a procedure which is possible in principle but very difficult in practice. The second, more serious, drawback involves the problem of empty cells for blocks that represent segments with low frequen- cies, such as at the extremes of a normal distribution. These shortcomings can be overcome when blocks are part of the experimental design, but are virtually insurmount- able after the fact. As methods for increasing experimental precision, the analysis of covariance and generalized randomized blocks models have been studied thoroughly by other inves- tigators but post hoc blocking methods have received virtually no systematic consideration. In View of their potential utility for incorporating concomitant variables into a design in order to increase precision or investi- gate nonadditivity of treatment effects (as in the case of regression heterogeneity), the remainder of this investi- gation is principally concerned with methods of post hoc block formation. 14 Scope of the Investigation The principal topic in the investigation of post hoc block formation methods is testing the hypothesis of null treatment effects when the data fit a normal theory model and are analyzed by an unweighted means analysis of variance. The choice of a test statistic is determined by considering, for the case of null treat— ment effects: (a) the expected values of mean squares when all assumptions are true; (b) the expected values of mean squares when the assumption of block-treatment additivity is violated; (c) empirical Type—I error rates when all assump— tions are met; and (d) empirical Type—I error rates when the assump- tion of block-treatment additivity is violated. Two secondary topics in the investigation are: (a) tests for block-treatment non-additivity (inter- action) and (b) the relative precision of the one-factor analy— sis of variance, the post hoc blocking methods, and the one-factor analysis of covariance, as well as the empiri- cal power of the one-factor analysis of variance and the 15 unweighted means analyses of the post hoc blocking methods for one case of noncentral treatments. The investigation is limited to a normally distri- buted dependent variable with constant variance across treatment groups and a joint bivariate normal density with a concomitant variable. Topics are investigated analytically where feasible or empirically with Monte Carlo techniques when analytic techniques are either unknown or unreasonably difficult. Where Monte Carlo techniques are required, cases are sampled from combi- nations of two and four treatment levels; two and four blocks; correlation of 0, 0.2, 0.4, 0.6, and 0.8 between dependent and concomitant variables; average block-treat- ment cell sizes between three and twenty; null and non- null treatment effects; and homogeneous and heterogene— ous regression. One further limitation should be noted. The analy- ses for the fixed- and sampled-range post hoc blocking methods are inapplicable if any block-treatment combi- nations have zero frequencies. For that reason, all investigations of the two methods are limited to cases where post hoc blocking resulted in non-zero cell frequencies. CHAPTER II DESCRIPTION OF THE INVESTIGATION The Normal Theory Model The basic model in this investigation is the classi- cal normal theory fixed effects model. Under the null hypothesis, the t treatment populations are the only populations of interest and, for each treatment popula- tion, the values of the dependent variable, Y, have a normal distribution with common variance 0Y2 and mean “y' The sample observations constitute independent ra- dom samples and treatment effects, if any, are strictly additive. The model is then ij =p + Bj + ejk lfooopt where j k = l,...,nj, the number of units in sample j. t H = % §=l“j, the average of t population means. Bj=uj-u t X B = 0 j=13 2 y)' and the ejk are independently N(0, o 16 17 The hypothesis of major interest is HO: “l: 000 = u = u or equivalently, The appropriate test statistic is the well known F ratio of mean squares for treatments and residual, as listed in Table 1. Table 1. Expected Values of Mean Squares in the One- Factor, Fixed Effects Analysis of Variance. Source of Degrees of Mean Expected Value Variation Freedom Square of Mean Square 2 t 2 o Treatments (t-l) MS + z n.3. T Y '=1 3 J J t 2 t-l Residual Z (n -1) MS 0 j=1 J R Y While a concomitant variable could have any kind of functional relationship with the dependent variable and be measured in any kind of metric, it is assumed for both theoretical and practical reasons that the con- comitant variable X and the dependent variable Y have a joint bivariate normal density with constant means, variances and correlation pxy in each treatment 18 population3. This assumption appears to be reasonable for much educational research and facilitates compari- son with earlier research of Cox (1957), Feldt (1958), and Pingel (1968), among others. Thus, it is assumed that for all t treatment popu- lations, the dependent and concomitant variables, Y and X, respectively, have a common bivariate normal density function with parameters u , u 2, o 2, and p . This 0 Yj Y X YX implies that the effects of treatment are additive and XI equal under the null hypothesis. The only alternative hypothesis entertained is a shift alternative, i.e., u # “y , for two or more pOpulations but all other Y 3 3 parameters are unchanged. The Generalized Randomized Blocks Model When blocks are added to the basic normal theory model, the design is as depicted in Figure 1. If blocks are considered fixed, as in the fixed-range post hoc block- ing method, then the n.. observations in the ith 1] and jth treatment group combination are assumed to be block drawn from a normal pOpulation with mean “ij and variance 2 0e. The fixed effects analysis of variance model is This assumption holds unless an exception is specifi- cally noted. One noncentral treatments case and one case of regression heterogeneity is included in the empiri- cal portion of this investigation. 19 , . + . . Yijk = U + Oi + Bj + (GB)13 ele where i = ll°°°'b; j: l'ooolt; k:l,..o,n.. 1] b t u = n.. = X Z pij; i=1 j=1 bt t a = n.. -u = 2 u-- -u; 1 1 j=1 1) t b = .. - = Z -- - U! 83 u 3 u i=lu12 b = ."' 0.0 + I (aB)ij “13 ul u J u e.. =Y ”U I ijk ijk ij. are independently N(0,c%2) for all i and j and Edi = fBj = ;(a8)ij = I(g8)ij = 0. 1 j l 3 When blocks are considered finite or random but treatment effects remain fixed, as may be the case in the sampled range post hoc blocking method, the generalized randomized blocks analysis of variance model becomes Yijk = p + ai + Bj + (a8)ij + eijk where i, j, k are defined above, 28. = 2(a8).. = 0; 3.33.1: are independently N(0, oez) for all i and j; 2 ai ~ N(O, 0a ); 2 0(a8)); and all random components are jointly normal. ijk 20 The hypothesis of principal interest in this set of experimental designs remains that of equal treatment means .31... t-l Note that block effects are assumed to be non-zero since HO: u.1 = ... = u.t or equivalently 28% = 0. blocking is included to increase precision, while the assumption of homogeneous bivariate normal distributions precludes non-zero interaction effects. In the generalized randomized blocks design, the apprOpriate ratio of mean squares for testing a treat- ment main effect depends on whether blocks are fixed or random (Scheffe', 1959, Collier, 1960), as well as whether subclass frequencies are equal, prOportional, or disprOportional. While many references, such as Anderson and Bancroft (1952) or Snedecor and Cochran (1967) treat unequal subclass frequencies as a nuisance due to accident or poor planning, the reason for the unequal frequencies determines the appropriate analysis, if any. Bancroft (1968), in a monograph on the topic of unequal subclass frequencies, points out that if the subclass frequencies are proportional to the marginal frequencies then the sums of squares remain orthogonal but a direct test of the null hypothesis may not exist since treatment and error mean squares have disparate expectations. The same appears to be true when unequal 21 population sizes require proportional sampling.4 If subclass frequencies are disproportionate due to treat- ment effects, population frequencies, etc., weighted least squares may be appropriate, while if the subclass frequencies would be prOportional (or equal in the cases of interest here) except for sampling fluctuation, an expected subclass frequencies analysis may be employed. In the expected subclass frequencies analysis, subclass means are multiplied by the corresponding expected fre- quencies and the resultant totals substituted into the usual proportional subclass analysis of variance formu- las. Table 2 contains the expected values of mean squares for the case of equal expected frequencies.5 Finally, if subclass frequencies are unequal only because of samp- ling fluctuation, (as is the case in the post hoc block- ing methods considered here) then the unweighted means analysis may be used, where all between classes sums of squares are computed on subclass means, then adjusted by the harmonic mean of the observed subclass frequencies, and within subclass data form the pooled estimate of residual variation as usual. Expected values for mean 4These conclusions were drawn from Wilk and Kempthorne (1956a, 1956b) but erroneously attributed to their 1955 paper. 5Other treatments of expected mean squares can be found in Wilk (1955) and Collier (1960) for randomized blocks designs, Cornfield and Tukey (1956) and the Wilk and Kempthorne series for factorial designs, and a generalized extension in Millman and Glass (1967). .mxooHQ mo soflumasmom map CH mxooHQ mo Hogans map u m .mmflosmsqmum Hamo Hmswm MOM m4 “coHummflumm>sfl mflnu mo msoflusnfluumflp msHHQEMm Hmofluflmfim on» as mm .>Hm> mmNHm Hawo cmnz same oflsoEnms mzu mo osam> pmuommxm msu Mo mwuflm Hamo mo u m msoflumcHQEoo poxflm How mammamcm mamme pmpnmflmzss may pH when mo same oflocaums opp “mammamsm mmflocmdqmnw mmmHUQSm pmuommxw mnu :fl HnH:HML o m nH o 0 mm: Ass. as Adamo cgnpflzv N N Hmsogmmm Afilnvfialuvs + w sofiuomumusH mm m ma 2 0 ma N be + N o w imaeww N m: 1H-QVIH-ue pamEummua n uxooam annm + mmomAMIC + mmo Ianu pm + N60 e N. mm m. 3 m2 3-3 3858? Hub u: + mu .m. O H a N m cum + o 5w m2 AHIQV mxoon m N m mxoon Eopsmm no muflcwm mxooam pmxfim mumnwm Eopmmum sofiumfium> smmz mo momummo mo monsom wsam> pmuommwm .Eopsmm mum muwso amusmsfluomxm .mxooam Eopcmm no .muflsflm .pmxflm can muowmmm usmEummHB pmxfim suflz cmflmmo mxooam pmNHEOpsmm pmuflamumsmw ecu as mmumsqm saw: mo modam> Umpommxm .N magma 23 squares for the unweighted means analysis of the genera- lized randomized block design with fixed treatment effects, fixed, finite or random block effects, and random experi- mental units are presented in Table 2. The relative efficiencies of the expected subclass frequencies and unweighted means analyses are believed to be quite simi- lar (Bancroft, 1968), so only the unweighted means ana- lysis is considered in the sequel. Inspection of Table 2 suggests that in the fixed effects model, under the null hypothesis, the expected mean square for treatments is 0:, the residual within cell population variance, while for the mixed effects model the expected mean square for treatments is 0: + 30:8, the residual population variance plus a component due to treatments by blocks interaction, if any. The appropriate F ratios for testing the hypotheses of equal treatment effects are, then, the mean square for treat- ments over the mean square for residual in the fixed effects model and the mean square for treatments over the mean square for treatments by blocks interaction in the mixed effects model. 7 If, however, blocks are a random sample from a finite population of blocks, the expected mean square for treat- ments under the null hypothesis is a: + (l-b/B)Oa§' where 8 denotes the number of blocks in the population of blocks. Neither the ratio of treatment over residual mean squares 24 nor the ratio of treatment over interaction mean squares has a central F distribution under the null hypothesis when interaction is present. That is, if blocks are finite, interaction is present, and central F distribution tables are used to define critical values for significance test- ing, then the actual probability of a Type-I error will be greater than the nominal a level in the table for the ratio of treatments over residual mean squares and less than the nominal a level for the ratio of treatments over interaction mean squares. In the sampled-range method blocks are defined by sample statistics and the variability of block endpoints may cause blocks to resem- ble a finite or random factor, rather than a fixed fac- tor as in the fixed-range method. Precision The precision of an experiment, as noted earlier, is inversely related to residual population variance and directly related to sample size. Thus, decreasing the population variance or increasing sample sizes yields a more precise design. Unfortunately, the best known and most useful attempts to quantify precision are refer- red to in terms of imprecision, the reciprocal of pre- cision. For that reason, the remainder of this section refers to measures and indices of imprecision. 25 Imprecision, the average variability of the difference between two treatment means, is essentially a function of the population variance and sample size. Following Cox (1957) the true average imprecision of a design is 2 Vt = Ave'V(Y. .. - Y’j")= Zoe/n, 37‘: the variance of the estimated difference between treat- ment means, averaged over all pairs of treatments where Y'j" Y.j'. = means for treatments j and j', a: = residual population variance for the design, and n = the number of experimental units per treatment. If it is assumed that a concomitant variable X has a homogeneous bivariate normal density with Y, then the conditional variance of Y for fixed X is denoted 2 2 2 o = o l - o y( oxy) where p denotes the population correlation between X and Y, and the minimum true average imprecision is then . _ 2 Min(Vt) - Zoo/n. An index of true imprecision, I can be formed from the t! ratio of true average imprecision for a design to the minimum true average imprecision, i.e., 2 Vt — Zoe/n I =————S-. _ t Mln(Vt Zog/n or, n O Q (Dh40lv 26 the ratio of the residual variance for a particular design to residual variance when all variance due to a concomi- tant variable is removed. In an actual experiment, the residual variance is estimated from sample data rather than being a known parameter value. Since the accuracy of the variance estimate depends on the number of degrees of freedom available for estimation, and that number varies with the type of experimental design, the index of impreci- sion can be modified to reflect that variability by use of Fisher's (1935) factor for information lost in esti- mation. The resulting index of apparent imprecision is I (f + 3) a t (f + l) 2 = oe(f + 3) 2 00(f + 1) where f = degrees of freedom for estimating 0:. The index of apparent imprecision was used by Cox (1957) and Feldt (1958) to study the effects of incorporating concomitant variables into experimental designs. So long as the total number of experimental units per treatment group remains fixed and the number of units per block- treatment cell is constant, the index of apparent impre- cision reflects the relative power of the F test statis- tics for each design. The power of the F statistic (for 27 t treatment groups) is a function of the degrees of free- dom for estimating residual variance and a non-centrality parameter t 2 K28. 2 1/2 (b = [ _L(t-1)/°e 1 n. in the one-factor design, Where k = fib in the generalized randomized blocks design, nb for equal cell frequencies. ¢i is simply the treatment mean square component for treat- ment effects divided by the residual variance. When an experimental design affects only the residual variance and its degrees of freedom, the index of apparent impre- cision is monotonically related to power. A Generalized Index of Apparent Imprecision: In Two of the post hoc blocking methods, the fixed- range and sampled-range methods, do not maintain equal cell frequencies. The non-centrality parameter for these methods is reduced by a factor of (ii/ml/2 from the case of equal cell frequencies, a reduction which is reflected in neither the index of true imprecision nor the index of apparent imprecision. Thus, neither index is directly applicable to the present investigation. It is clearly desirable to use an index which is identical to the index of apparent imprecision when cell frequencies are equal, yet reflects the loss of power inherent in the disparate 28 cell frequencies of the unweighted means analysis. Consider the generalized index of imprecision f+3 ° f+1 H II e~|<> whooto t anB? where ¢o = (t—l,/Oo' the squared non-centrality parameter for equal cell frequencies and minimum residual variance 2 00; t2 2 fibZB. ¢a = (t-l /09, the squared non-centrality parameter for a design with non-zero cell frequencies and actual residual variance 0:, and %$%-= Fisher's adjustment for estimation of 0:. Thus, t anBg = t-l /°o f+3 g t 2 ' f+l fibZB. t-l /Ge no (0N f+3 ' f+ :12 ON H 0' and if cell freqeuncies are equal-~as in the one-factor design, analysis of covariance for the one-factor design, and the post hoc blocking within treatments method--then fl is identical to n and the generalized index Ig reduces 29 to the index of apparent imprecision used by Cox and Feldt. Precision of the One- and Two-Factor Designs Inspection of the expected mean squares in Tables 1 and 2 for the one- and two-factor designs suggests that the relative precision of the two types of designs is a function of the relative magnitudes of gyz, the residual variance of the one-factor design without block- 0 o v 0 ~ 2 ing or covariance adjustment, the ratio n/n, and 0 or e oez + (1-b/B)noa§ in the two-factor design with fixed, finite, or random blocks, respectively. Since 0 2 = a8 0 under the assumption of homogeneous regression across treatment populations, that component is not considered further. The imprecision of the one-factor, completely ran- domized design is a straightforward function of the sam- ple sizes and the correlation between the dependent vari- able and the (unused) covariable. The unadjusted average variance of treatment mean differences is _ 2 Vt - 20y /n, while the minimum average variance is . _ 2 Mln(Vt) - 200 /n. 2 _ 2 2 but do — (l p )0y and thus the generalized index of apparent imprecision 30 is 19 = ?:_:—;§T . f + 1 Thus, when p2 = 0, the one-factor, completely randomized analysis of variance is the most precise design available, with maximum degrees of freedom for estimating the resi- dual variance. Clearly, too, as p2 becomes large, this design is increasingly less precise and blocking or covari- ance adjustment becomes more advantageous. Cox (1957) showed that when the analysis of covari- ance is applied to the one-factor design, the index of apparent imprecision Since the number of experimental units per treatment is not altered, the generalized index I9 is identical to Ia' In the analysis of covariance, Ig has the interesting property of being independent of all parameters but the residual degrees of freedom. As is demonstrated in the following sections, it is the only index considered in 31 this paper which does not vary with the correlation between dependent and concomitant variables. The relative imprecision of the two-factor or genera- lized randomized blocks design is rather less straight- forward than for the one-factor design, depending on the within block variability of the concomitant variable, as well as p and the sample sizes. Feldt (1958) has shown that given the assumptions of bivariate normal densities, homogeneous regression and homoscedasticity, the variance of the dependent variable in block i is 2 o . 2 2 2 Xi o = o l - l - Y1 YI 0( O2)] x where 0x2 = variance of X in block i. 1 Since oez is simply the average of oyz over blocks, 2_ 2_2_—2 2 0e - Cy [l p (1 Ox /Ox )1 where E¥2 denotes the average within block variance of X. The generalized index of apparent imprecision is then 32 Clearly, as 3&2, the within block variance of the con- comitant variable, approaches 0, the residual variance for this design approaches 002, its minimum value and, as 3&2 approaches 0X2, the residual variance approaches 2 . . o , its maXimum value. Y Alternatively, if 0x2 is considered to be the sum of between and within block variances, 0x2 = 0:2 + 3&2, then maximizing oiz, the between blocks variance of X, maximizes the precision of the design if all other fac- tors are held constant. Conversely, small between block variance of the concomitant variable leads to a less precise design. Thus, it is desirable to have both a strong correlation between Y and X and minimum average within blocks variance on X. Within Block Variation of the Concomitant Variable The residual variance, 082, as noted in the previous section, is a function of the correlation with the con- comitant blocking variable and the concomitant variable's average within block variance, while the generalized index of apparent imprecision is also affected by the residual degrees of freedom and the variability of cell 2 2 sizes. Since 0y , Ox , and p are assumed to be fixed for any population or set of populations, the generalized 33 index of imprecision and thus the precision of the three methods of block formation depends on 3&2, the within blocks variance of X, as well as the number of degrees of freedom available for estimating 082 and the harmonic mean of cell frequencies. Thus, the value of 3¥2 plays an important role in this investigation. The first method of post hoc block formation, post hoc blocking within treatments, forms equal sized blocks within samples on the concomitant variable. For this . . . 2 . . method, the Within blocks variance, Ox , is a function of order statistics.6 Within any one treatment group j, S 3-1 s % “X0021 ' 2 “X00 xuml o 2 = k=r n(n-l) k=r k'=r+l xij for n > 1, 2 2 _ E[X(k) ] - E[X(k)] for n — 1 where n = number of units per block-treatment combination, block number, i r (i-l)n, s = i(n), X(k) = kth order statistic, i.e., the kth ordered value in the sample of size nb = N/t. All treatment groups are random samples of the same size 6The following is adapted from Pingel (1968). 34 from populations with identical distributions on X and thus the within block variance, OX:" is constant across treatment populations for each bloc; i and equal to the marginal within block variance oxi. The average within block variance is then —2_1 2 0x - bzcxi , the average over blocks of the marginal within block variances. The expected values of X(k)’ X(k)2' and X(k)x(k') have no simple expression, but tables of numerical approxi- mations for normal deviates from samples of size 20 or less have been computed by Teichroew (1956) and repro- duced in Sarhan and Greenberg (1962). Except for small sample sizes, Monte Carlo estimation is the most feasible approach. For the fixed—range post hoc blocking method, where all experimental units are assigned to blocks according to the b-l population quantiles of X (median, quartiles, etc.), the marginal within block variance of block i is O 2 = 1 + X(i-1)f‘xHqupalsoz .m I m moabme Eoum pmusoeoo mommo o>HuH©©¢ we. NH. mm. No.s mm. mm. mem2\ems sm.s mm. os.s No.H am. am. mmz\emz scrum: mes mmcmmupmsmsmm he. so. 4m. mo.s mm. oo.s memz\ems mo.s «a. oo.s No.a em. oo.H mm2\ems ponpmz mmm mmcmmupmxsm mo.H om. oo.H mo.a om. ma. mms\emz smflmmo Houommuwso mmsmm :mflpmz mmcmm spews: ofiumm x~.o M as mmmo m>sussnsuspsa o.o Inmamamm can Inmxflm obs pan smflmmo HouOMMIoGO .mponumz mcfixoon 00m umom mmsmm ”msoHuHosou m>euflpp<1soz can o>Hu watts mnu CH mmumsom one: mmmum>¢ mo moflumm man no momcmm was msmflomz .ma manna 66 0.71, with most of the variability due to p rather than variations of b and t at each level of p. These results, while not definitive, suggest that the residual mean square is the apprOpriate error term for testing the null-treat- ment effects hypothesis. For the SRPB method, the average treatments mean square was highly variable across combinations of p, t, and b. The ratio of average mean squares MST/MSR had a median of 1.1 and a range of 0.95, nearly nine times as variable as the additive case. The average mean squares ratio MST/MSTB had a median of 0.25 and a range of 0.65. These results, when compared to the additive condition results, suggest that MST contained some fractional com- ponent due to block-treatment interaction, as would be the case if blocks represented a finite factor. Empirical F Distributions While the average values of mean squares are of some theoretical interest, for practical purposes the degree of fit between empirical and reference distribu- tions is far more important. In particular, if a ratio of mean squares has empirical frequencies close to theo- retical frequencies at the usual o levels (0.10, 0.05, 0.01) in both the additive and non-additive conditions, then that ratio and the design for which it was computed is a candidate for actual use. If it shows high power, 67 relative to logical alternative designs, then it should be put to use. Table 14 presents empirical Type-I error rates of the central F ratio MST/MS for the one-factor design, R with the medians and ranges of empirical frequencies at the bottom of the table. These distributions were included because the one-factor design is the basic design which the post hoc blocking methods modify and because it provided both a check on the accuracy of data genera- tion and some indication of the amount of variability to be expected in the empirical distributions. The median error rates tended to be slightly conservative, i.e., with only one exception the median empirical error rates were less than the theoretical error rates of the reference distributions. The range of error rates varied from sixty-eight at the fiftieth percentile to thirteen at the ninety-ninth percentile, with no apparent trends across combinations of t, b, n, or p. The variability and lack of exact fit between empirical and theoretical distributions was most likely because the empirical dis- tributions were based on a finite number of samples, while the theoretical distributions assume an infinite population of samples. Empirical Type-I error rates under the null hypothe— sis for the central F ratio MST/MSRI additive condition, are presented in Tables 15 and 16 for the FRPB and SRPB methods, respectively. For both methods, median observed Table 14. 68 Observed Type-I Error Rates for the Ratio MST/MSR: One-Factor Design, Additive Condition, ¢ = 0.0 Expected Rate (1000o) p t N* dfl df 750 500 250 100 50 10 0.0 2 10 1 18 740 491 238 90 38 10 2 20 1 38 735 451 227 89 43 7 4 10 3 36 752 519 248 97 41 8 4 12 3 44 733 500 241 83 33 3 4 20 3 76 742 493 245 100 58 16 4 40 3 156 765 494 248 101 41 5 0.2 2 20 1 38 734 491 253 82 46 9 4 20 3 76 743 500 238 92 45 9 0.4 20 l 38 754 473 234 101 56 12 20 3 76 740 491 258 107 56 11 0.6 2 10 1 18 749 509 265 97 51 10 2 20 1 38 765 491 250 97 53 10 4 10 3 36 750 486 235 100 54 13 4 12 3 44 744 481 244 90 44 6 4 20 3 76 748 496 242 89 49 15 4 40 3 156 735 496 243 91 43 11 0.8 2 20 l 38 736 484 249 103 51 11 4 20 3 76 750 500 269 108 62 14 Median 743 492 245 97 48 11 Range 32 68 42 26 29 13 *N = bxn in the Post Hoc Blocking Methods. 69 Table 15. Observed Type-I Error Rates for the Ratio MST/MSR: Fixed-Range Post Hoc Blocking Method, Additive Condition, ¢ = 0.0 Expected Rate (1000o) p t b n df df 750 500 250 100 50 10 0.0 2 2 5 l 16 725 478 236 97 50 6 2 4 5 1 32 749 484 247 90 46 10 4 2 5 3 32 739 509 242 99 38 9 4 4 3 3 32 748 481 255 97 45 9 4 4 5 3 64 736 512 258 108 53 12 4 4 10 3 144 764 488 253 88 37 9 0.2 2 4 5 l 32 752 502 239 90 42 7 4 4 5 3 64 758 487 233 107 58 8 0.4 2 4 5 1 32 755 501 237 98 56 6 4 4 5 3 64 739 483 239 111 58 15 0.6 2 2 5 l 16 760 500 256 109 54 8 2 4 5 1 32 750 505 248 93 55 15 4 2 5 3 32 747 497 246 99 47 12 4 4 3 3 32 771 493 262 109 60 16 4 4 5 3 64 740 495 259 95 61 18 4 4 10 3 144 758 519 263 100 49 10 0.8 4 l 32 751 508 262 100 55 14 4 4 5 3 64 744 494 245 105 59 13 Median 750 496 248 100 54 10 Range 32 58 42 26 29 13 70 Table 16. Observed Type-I Error Rates for the Ratio MST/MSR: Sampled-Range Post Hoc Blocking Method, Additive Condition, 9 = 0.0 Expected Rate (lOOOo) p t b n df1 df2 750 500 250 100 50 10 0.0 2 2 5 1 16 741 487 235 92 37 8 2 4 5 1 32 744 484 236 87 40 5 4 2 5 3 32 746 495 256 100 42 8 4 4 3 3 32 769 496 273 86 36 9 4 4 5 3 64 743 510 253 111 58 15 4 4 10 3 144 773 495 251 83 43 5 0.2 4 5 1 32 751 500 242 86 41 7 4 5 3 64 744 492 244 104 50 6 0.4 2 4 5 1 32 731 481 252 99 56 12 4 4 5 3 64 750 489 239 109 57 18 0.6 2 2 5 1 16 738 498 250 103 54 10 2 4 5 1 32 745 519 241 89 51 14 4 2 5 3 32 752 495 255 108 53 11 4 4 3 3 32 762 513 259 111 57 18 4 4 5 3 64 739 507 257 97 50 12 4 4 10 3 144 761 505 258 112 43 7 0.8 2 4 5 1 32 761 519 241 109 57 12 4 4 5 3 64 762 507 268 113 48 6 Median 750 497 252 102 50 10 Range 38 38 38 30 22 13 71 error rates fluctuated around the expected error rates with variability comparable to that observed for the one-factor design reported in Table 14. The variability in error rates revealed no apparent trends across com- binations of t, b, n, or p. Empirical Type-I error rates for the central F ratio MST/MS additive condition, are presented in Tables TB' 17 and 18 for the FRPB and SRPB methods, respectively. The median observed error rates for both methods were close to expected error rates for both methods, with the fixed-range method's error rates slightly more con- servative for a g 0.25. Variability in error rates was comparable to the variability for the one-factor design in Table 14. As was the case for Tables 14 — 16, there was no apparent trend associated with changes in t, b, n, or p. The observed Type-I error rates in the null treat- ments, additive condition were consistent with the empiri- cal average mean squares in Tables 6 - 9. Both of the ratios MST/MSR and MST/MS had empirical a levels close TB to nominal a levels for both the fixed- and sampled-range post hoc blocking methods. A clear choice of post hoc blocking method and test statistic depends on the effects of violating the assumption of non-additivity and the precision of the method. The importance of including non-additive cases can be clearly illustrated by the example of Pingel‘s (1968) 72 Table 1?. Observed Type-I Error Rates for the Ratio MST/MSTB: Fixed-Range Post Hoc Blocking Method, Additive Condition, ¢ = 0.0 Expected Rate (10008) p t b n dfl df2 750 500 250 100 50 10 0.0 2 2 5 1 l 718 482 237 85 41 4 2 4 5 1 3 743 488 234 88 51 11 4 2 5 3 3 758 494 243 89 44 13 4 4 3 3 9 752 502 248 90 44 9 4 4 5 3 9 743 508 241 96 50 8 4 4 10 3 9 766 493 241 93 47 7 0.2 2 4 l 3 777 510 219 74 31 6 4 4 5 3 9 757 503 244 89 38 7 0.4 2 4 5 1 3 755 491 243 97 43 7 4 4 5 3 9 754 510 257 99 54 9 0.6 2 2 5 1 1 758 500 247 96 42 6 2 4 5 1 3 771 506 232 84 42 11 4 2 5 3 3 775 523 247 100 57 9 4 4 3 3 9 780 515 257 90 42 7 4 4 5 3 9 743 500 243 83 38 6 4 4 10 3 9 776 513 270 112 48 7 0.8 2 4 l 3 764 523 271 119 61 10 4 4 5 3 9 751 498 254 105 50 8 Median 758 502 244 92 44 8 Range 38 41 51 45 30 9 73 Table 18. Observed Type-I Error Rates for the Ratio MST/MSTB: Sampled-Range Post Hoc Blocking Method, Additive Condition, ¢ = 0.0 Expected Rate (lOOOo) p t b n df1 (if2 750 500 250 100 50 10 0.0 2 2 5 1 1 734 476 226 78 34 ll 2 4 5 l 3 738 464 230 91 54 14 4 2 5 3 3 754 491 252 102 55 10 4 4 3 3 9 772 517 246 90 42 8 4 4 5 3 9 730 496 258 100 48 10 4 4 10 3 9 776 486 258 94 45 8 0.2 2 4 5 1 3 747 497 229 78 24 5 4 4 5 3 9 758 499 225 91 41 15 0.4 2 4 5 1 3 724 489 236 99 45 12 4 5 3 9 740 494 239 95 52 13 0.6 2 2 5 1 l 746 502 246 95 46 9 2 4 5 1 3 751 505 256 97 51 11 4 2 5 3 3 771 517 249 93 46 12 4 4 3 3 9 756 516 246 86 49 6 4 4 5 3 9 737 477 265 96 49 13 4 4 10 3 9 758 516 263 114 55 10 0.8 2 4 5 1 3 772 517 269 123 70 13 4 4 5 3 9 757 505 247 99 55 8 Median 752 498 246 96 49 10 Range 52 53 44 36 46 10 74 investigation of a priori block formation methods. He considered only the case of block-treatment additivity in his Monte Carlo study and concluded that the sampled- range method had a random blocks factor and thus the apprOpriate test of the null treatment effects hypothe- sis in the ratio of mean squares for treatments and block- treatment interaction, MST/MSTB' However, the tables in Appendix B show that under conditions of severe non- additivity the average mean square for treatments was much less than the average mean square for interaction and the ratio MST/MS was extremely conservative. The TB ratio MST/MSR provided a better, but slightly liberal fit to the theoretical F distribution. Empirical Type-I error rates for the ratio MST/MSR under conditions of non-additivity are presented in Table 19 for the fixed- and sampled-range post hoc blocking methods. For the FRPB method, observed and empirical error rates were in close agreement over all combinations of p, t, and b. The median observed error rates were close to theoretical error rates and the ranges were com- parable to those of the additive cases. Results for the SRPB method were quite different. For that method, the observed error rates became increas- ingly liberal as p varied from 0.2 to 0.8. Error rates tended to be less liberal for t = 4 than for t = 2, which was due more to the larger pooled sample sizes and the method of generating non-additive cases than the change 75 mm mea mmH mow wH ow mm om mosmm ma me me mew Ha om ooa mew cmflpmz em oma mww moe m me woa mew spate: mm maa mma mmm m me eoa mew em m m e e an nma flew mee m om How wmw wm H m e w m.o we we weH eom ma Hm eoa mew swepmz ma we weH mam ma om owa Hmw em m m e e wm hm mmH eom ma om eoa Hmw wm H m e w ma mm mwa mew ma Hm ooa mew wm m m w e w.o ma mm mHH mmw Ha om mm mmw spate: ma mm mad cow Ha om mm omw em m m e e ma mm me mmw HH Hm mm mew wm a m e w e.o m ee wm wmw m ee Hm wmw swaps: wa om em Hmw Ha me am omw em m m e e m mm mm wew m oe Hm mew wm H m e w w.o oa om ooa omw ow om ooa omw mp mp : b u a Asoooae mpmm emuommxm Asoooae mums empommwm bonus: msflxoon oom oonumz msflxoon pmom mmsmmlpwHoEmm oom umom mosmmlpmxflm o.o u e .coHuflpsou m>flueppmlsoz .mtonumz mcfixoon oom umom mmsmm somamamm can upmxflm "mmz\emz oflpmm map How mmumm uouum Hlomxe pm>ummno .ma manna 76 in the number of treatment groups. Median observed error rates were clearly greater than the theoretical error rates and were far more variable than any of the cases in Tables 14 - 18. Under conditions of extreme non-addi- tivity (p 3 0.4), actual a levels were higher than the nominal a levels of the theoretical F distribution. Table 20 contains non-additive condition, empiri- cal Type-I error rates for the ratio MST/MSTB for both the fixed- and sampled-range post hoc blocking methods. For both methods, the observed error rates were conser- vative with p = 0.2 and rapidly approached zero as p increased. The SRPB error rates tended to be slightly less conservative than those of the FRPB method but not. to any appreciable degree. For all cases generated with p > 0, the SRPB ratio MST/MSTB was more conservative than the corresponding ratio MST/MSR was liberal. The empirical average mean squares and the empiri- cal F distributions support the conclusion that blocks represent a fixed effect in the fixed-range post hoc blocking method and that the ratio MST/MSR has a distri- bution reasonably close, if not identical to, the central F distribution, even under conditions of extreme non- additivity. Blocks in the sampled-range method, however, appeared to behave like a finite factor. The factor is clearly not finite in the usual sense of a random sample from a finite population, but rather each sample of blocks 77 e mw em omH b om Hm whH mmcmm H w w OH 0 H w wH smemz o o o H o o o H ngbmz o o o o o o o H m m m e e o o o w o o o H m H m e w m.o o o H mH o o H m smemz o o o e o o H w m m m e e o o H OH o o H m m H m e w w w w mH o H w wH m m m w e w.o w m eH om o e 5 mm smHowz o w m me o w e we m m m e e e m mH en 0 n n em m H m e w e.o w Hw wm mnH e ew me mmH smemz H eH em emH H mH he mnH m m m e e e mw Hm mmH n om Hm mmH m H m e w w.o OH om OOH Omm OH Om OOH Omm New How a p p O HUOOOHV mumm touommxm ASOOOHV spam pmuommwm bonus: msHHOOHm oom bonus: msHMUOHm umom mocmmlpmHmEmm oom umom mmcmmlthHm o.o n e .coHqucoo o>Hqup4Isoz .mponuoz mstoon oom pmom mmsmm upmHmEmm was upmme "mem2\emz oHumm may you mmumm uouum leoma pm>ummno .ow mHnma 78 represents a slightly different set of b blocks, since block endpoints are defined by the b-l quantiles of the pooled X observations. In small samples the block end- points are relatively variable and the blocking variable behaves as though the population of blocks is larger than b. As sample sizes increase, the pooled sample quantiles become less variable and the blocking variable behaves as though the size of the block population approaches b. In very large samples the blocks would be fixed, for all practical purposes, and results should be similar to those of the fixed-range post hoc blocking method. For the relatively small sample sizes in this inves- tigation, as the degree of non-additivity increased, the average treatments mean square became larger than the average residual mean square, yet smaller than the average interaction mean square. Similarly the ratio MST/MSR became increasingly liberal as the interaction increased, while the ratio MST/MSTB became extremely conservative. Both ratios appeared to have central F distributions under the null hypothesis in the additive condition. Block-Treatment Non-Additivity Block-treatment additivity has been treated as one of the assumptions of the model and previous discussion has focused on what effect the violation of that assumption 79 had on the tests of the null treatments hypothesis. If, however, the primary motivation for post hoc blocking is to investigate non-additivity or as suggested earlier, there is a desire to use the sampled-range method when non-additivity is suspected, then additivity of block and treatment effects should be considered as an hypothe- sis to be tested, rather than an assumption which may have been violated. The remainder of this section refers to the null hypothesis of block-treatment additivity as simply the null hypothesis unless it is unclear which null hypothesis is involved. The two conditions in the empirical investigation were additive block and treatment effects, null treatments and non-additive block and treat- ment effects, null treatments. Table 21 contains empirical Type-I error rates for the ratio MSTB/MSR of the fixed- and sampled-range post hoc blocking methods under the null hypothesis. The median observed error rates were close to theoretical values for both methods. The variability in observed error rates was consistent with that of the additive cases in Tables 14 - 18 and there were no consistent trends across vari- ations in t, b, or p. Observed and nominal 8 levels were in close agreement when testing the additivity hypothesis with the ratio MSTB/MSR. Table 22 contains the observed power of the ratio MSTB/MSR for the FRPB and SRPB methods under conditions of non-additivity. For both methods, power increased 80 Table 21. Observed Type-I Error Rates for the Ratio MSTB/MSR: Fixed- and Sampled-Range Post Hoc Blocking Methods, Additive Condition, ¢ = 0.0 Fixed-Range Sampled-Range Method Method Expected Rate Expected Rate (1000a) (1000o) p t b n dfl df2 250 100 50 10 250 100 50 0.0 4 2 5 3 32 252 96 45 7 251 93 52 ll 2 4 5 3 32 260 112 57 10 272 97 44 8 4 4 5 9 64 228 104 60 19 265 99 53 14 0.2 2 4 5 3 32 265 110 46 10 273 106 50 10 4 4 5 9 64 239 94 51 9 240 106 53 9 0.4 2 4 5 3 32 253 117 65 17 252 98 54 12 4 4 5 9 64 231 104 61 17 248 102 55 12 0.6 4 2 5 3 32 238 113 40 10 241 100 57 15 2 4 5 3 32 243 91 45 10 256 96 50 ll 4 4 5 9 64 242 117 59 11 273 114 61 16 0 8 2 4 5 3 32 248 104 54 10 256 98 57 12 4 4 5 9 64 246 103 55 15 264 108 61 12 Median 245 104 54 10 256 100 54 12 Range 37 26 25 12 33 21 17 8 81 OOO OOO OOO OOO OOO HOO OOO HHO mOcmm NOO OHO OHO OHO OOO OHO ONO ONO cmHOmz OOO OOO OOOH OOOH OOO OOOH OOOH OOOH cmems OOOH OOOH OOOH OOOH OOOH OOOH OOOH OOOH OO O O O O OOO OOO OOO OOOH OOO OOOH OOOH OOOH NO O O O N O.O OOO OOO OOO OOO OOO OOO OOO NOO OOHOmz OOO NOO OOO OOO OOO HOO HOO OOO OO O O O O OOO OOO OOO OOO OOO OOO OOO NOO NO O O O N NOO OHO OHO OHO OOO OHO ONO ONO NO O O N O O.O OON OOO ONO OOO NON NOO OOO NOO OOHOmz OON NOO OOO HOO OON OOO HOO ONO OO O O O O OHN OOO OOO OOO HON ONO OOO OOO NO O O O N O.O OO OOH ONN NNO OO OOH OON OOO cmHOmz OO OOH ONN OHO OO ONH OON ONO OO O O O O HO OOH ONN ONO OO OOH NHN OOO NO O O O N N.O OH OO OOH OON OH OO OOH OON NOO HOO a n H O AOOOHxV Hm>OH a tonne: mossmupmHmEmm AOOOHxV HO>OH o spasms mocmmuemme pmom mmsmmlpmHmEmm pom Iowam o.o e .soHUHpcoo m>Hqupmlsoz .mpozumz mconon oom “mmz\memz oHumm me» up speed em>ummno .NN anme 82 rapidly as p varied from 0.2 to 0.8, with the SRPB method slightly, but consistently, more powerful. At least for the specific cases investigated, the ratio MSTB/MSR was quite sensitive to non-additivity. For example, with p = 0.6 and d = 0.5, the median observed power was 0.864 and 0.870 for the FRPB and SRPB methods, respectively. This is the same value of p for which the ratio MST/MSR was unreasonably liberal in Table 19. It appears that the ratio MSTB/MSR provides a reasonably powerful prior test for non-additivity at the point where the non-addi- tivity is contaminating the test for null treatment effects with the ratio MST/MSR. Precision Precision, as shown in Chapter I, is one of the cen- tral topics in experimental design. For a fixed number of treatments and fixed numbers of experimental units per treatment group, the most precise design provides the most powerful test of the hypothesis of no treatment effects. The generalized index of apparent imprecision Ig' which is used in this investigation is the ratio of the squared non-centrality parameter for an ideal design--minimum residual variance and equal cell frequencies--to the squared non-centrality parameter for the design under considera— tion, modified to reflect the number of degrees of freedom 83 available to estimate residual variance. If the parti- cular design of interest has equal cell frequencies, the generalized index is identical to the better-known index of apparent imprecision used by Cox (1957) and Feldt (1958), for example. For both indices, large values indicate a relatively imprecise design; the closer the index approaches the lower limit of 1.0, the more precise the design. Equivalently, the design with the numerically lowest index of imprecision provides the most powerful test of the null hypothesis. Comparisons of precision between the one-factor design, one-factor analysis of covariance, and the fixed- and sampled-range post hoc blocking methods are most appro- priately made within each combination of treatments, blocks, average cell size, and correlation between dependent and concomitant variables. Within each combination, Iq is a function of: (a) variation in cell frequencies as reflected in a, the harmonic mean of the cell frequencies; (b) the degrees of freedom available for estimating residual variance; and (c) the magnitude of the residual variance, a func- tion of the concomitant variable's within-block variance. It was expected, from prior research on a priori block- ing methods, that within block variance of the concomi- tant variable would be smaller for the fixed-range method, contributing to greater precision. 84 Inspection of Tables 4 - 7 reveals that without excep- tion the concomitant variable's residual variance was smaller for the fixed-range method than for the sampled- range method. However, the same tables show that the average harmonic mean of cell frequencies was always less for the fixed-range method, which contributes to lesser precision. Table 23 presents indices of imprecision for the four methods for correlations of 0.0 and 0.6 and selected combinations of treatments, blocks, and average cell sizes. The table also presents empirical non-central F distri- butions (power) for the one-factor design and the fixed- and sampled-range methods where blocks and treatments are additive. They were included to provide verification of conclusions drawn from the generalized index of impre- cision. For p = 0.0, the one-factor analysis of variance had the lowest value of Ig, followed by the one—factor analysis of covariance, the sampled-range method, and finally, the fixed-range method. In one case, the sam- pled-range and analysis of covariance indices exchanged places in the rank ordering. This was probably due to the fact that the indices were quite close in value and the sampled-range index was computed from empirical esti- mates while the analysis of covariance index was computed directly from the residual degrees of freedom. For p = 0.6, the one-factor analysis of covariance was most pre- cise (lowest value of 19), followed by the sampled-range 85 OHO H - - - - OHO H - - - - OOH O O>OO2O NOH.H HOH OOO OOO OOO NOO.H OO NON OOO OOO OOH O OOOO OOH.H NOH OOO OOO NOO OOH.H OO NON HOO OOO OOH O OOOO NOO.H OOH OON NOO OOO OHO.H NO OON NOO NOO OOH O OOHOOO-O:O OH O O OOO.H - - - - OOO.H - - - - ON O <>OO2O OON.H OOH HOO OOO OOO NOH.H OO OHN ONO HOO OO O OOOO OOO.H ONH OHO OOO OOO OON.H ON OHN ONO HOO OO O OOOO OOO.H OO OON OOO NOO ONO.H NO OON OOO OOO ON O HOHOOO-OOO O O O NOO.H - - - - NOO.H - - - - NO H <>OOz< HON.H OO NOH NON HOO OOH.H OO OOH OON OOO NO H OOOO OOO.H NO HOH NON ONO ONN.H OO NOH OON NNO NO H OOOO OOO.H OO HOH OON OOO NOO.H OO HOH OON OOO OO H OOHOOO-OOO O O N NOO.H - - - - NOO.H - - - - OO O <>Ouz< OOO H OO ONN HOO NHO NOH.H NO OOH OHO ONO NO O OOOO OOO.H NO OON OOO OON HON.H ON OOH OON, OOO NO O OOOO NOO.H OO OOH NOO OON OOO.H OO OOH NOO OHN OO O HOHOOO-OOO O N O HOH.H - - - - HOH.H - - - - NH H <>Ooz< ONO.H NN OHH OON OOO NNH.H OH NO HOH ONO OH H OOOO OOO.H OH ONH ONN OOO OON.H OH OO OOH OOO OH H OOOO NNN.H OH NO HON OOO OOO.H OH OO NOH OOO OH H HOHOOO-O=O O N N OH OH OO OOH OON OH OH OO OOH OON NOO HOO Oocsmz O O p HOOOHOO HO>OH O O.O u o HOOOHOH.HO>OH O o.o u a HOOHHHOOOO o>HHHOO< o6 .o.o n O No. ":onHomudsH mo OOUHpsH HOuHuHoem pom mmz\emz oHpOm 6:» mo sexed po>nombo H n e .O «Home 86 method, then the fixed-range method, and finally, the one-factor analysis of variance. The ordering was con- sistent across all combinations of t, b, and n. Table 24 presents the same indices of imprecision and power for five values of p(0.0, 0.2, 0.4, 0.6, 0.8), four treatments, four blocks, and an average cell size of five. In both Tables 23 and 24, the analysis of covari- ance was most precise for p 3 0.2 and the sampled-range method was always more precise than the fixed-range method. Neither post hoc method was more precise than the one-factor analysis of variance until p reached 0.6 and then both methods were more precise. These results were confirmed by the non-central F distributions. In all cases lower values of Ig were associated with greater power. The relative precision of the two post hoc blocking methods could be improved by increasing the number of blocks and/or the number of experimental units, but how much is unclear. Simply increasing the number of blocks without also increasing the number of experimental units quickly reaches a point of diminishing returns. Tables 4 - 10 include the number of additional iterations required to generate 1000 cases with non-zero cell frequencies for each design. Consider just the case of four treat- ments and four blocks: for n = 3, the fixed- and sampled- range methods required over 700 and 300 extra iterations, respectively; for n = 5, they required roughly 50 and 20 extra iterations, respectively; and for n = 10, no extra 87 Table 24. Observed Power of the Ratio MST/MSR and Empiri- cal Indices of Imprecision: Additive Case, ¢ = 1.0, t = 4, b = 4, n = 5 ‘ a Level (x1000) p df df Method I l 2 250 100 50 10 g 3 76 One-Factor 594 369 249 97 1.026 0.0 3 64 FRPB 551 326 213 75 1.250 3 64 SRPB 551 325 218 80 1.192 3 76 One-Factor 596 393 272 98 1.069 0.2 3 64 FRPB 572 335 227 85 1.262 3 64 SRPB 590 367 245 99 1.198 3 76 One-Factor 596 359 246 89 1.221 0.4 3 64 FRPB 594 368 238 81 1.282 3 64 SRPB 602 373 248 92 1.223 3 76 One-Factor 597 369 255 98 1.603 0.6 3 64 FRPB 655 439 314 124 1.348 3 64 SRPB 650 449 331 136 1.286 3 76 One-Factor 586 354 235 90 2.850 0.8 3 64 FRPB 771 588 449 211 1.551 3 64 SRPB 792 617 459 214 1.486 Analysis of Covariance Index of Imprecision 1.040 FRPB = Fixed-Range Post Hoc Blocking Method SRPB = Sampled—Range Post Hob Blocking Method ANCOVA = Analysis of Covariance, One-Factor Design Data for p = 0.0 and 0.6 are reproduced from Table 23. 88 iterations were required. Reducing either the number of blocks or treatments helped but clearly, small average cell sizes is likely to involve empty cells, in which case the post hoc methods are inapplicable. Comparison to A Priori Block Formation One final consideration: how much precision is lost by forming blocks after the fact rather than incorporating blocks directly into the experimental design? Pingel (1968) provides the estimates of concomitant variable within-block variance necessary to compute indices of imprecision for some of the specific combinations of design parameters in the present investigation. Indices of impre- cision for the fixed- and sampled-range methods, both a priori and post hoc, are presented in Table 25. Indices of imprecision for the one-factor analyses of variance and covariance are repeated from Tables 23 and 24 for ease of comparison. Without exception, the a priori methods had smaller values of 19. In fact, the a priori block- ing methods were more precise than the one-factor analy- sis of variance for p 3 0.2,while the post hoc blocking methods were more precise only for p 3 0.6. The analysis of covariance was the most precise of all methods for p 3 0.4. Within block variances were virtually identi- cal for the respective a priori and post hoc methods, 89 .HmmmHv Homon an pmusmeoo OOCMHHM> xooHQIanuH3 monum>m memHum> HomuHEoosoo mo momeHumm Eoum GOUMHsono moonumfi HHoHum < “Om mooHch oeo.H omm.w mmw.H mme.H mmw.H Hmm.H m e e m.o oeo.H mmw.H wHH.H mmw.H wHH.H mem.H m e e wmo.H mem.H eeH.H Hmw.H eeH.H mom.H m e w hmo.H OeO.H mmw.H eoe.H mmw.H mee.H m w e HmH.H nwh.H mmm.H mwe.H mem.H mme.H m w w w.o oeo.H wa.H mmo.H mww.H mmo.H wmw.H m e e e.o oeo.H OOO.H Omo.H mmH.H Omo.H wow.H m e e w.o oeo.H mwo.H Hmo.H me.H Hmo.H mmw.H m e e wmo.H wmo.H Hmo.H mmH.H Hmo.H mmw.H m e w bmo.H emo.H Hmo.H wOH.H Hmo.H How.H m w e HmH.H mmo.H mHH.H eeH.H mHH.H emw.H m w w 0.0 HHoHHm m oom umom HHOHHm m oom pmom s A u a d>OUZ¢ <>Oz< mmsmmlomHmEmm mmgmmlthHm mosmHum>ou pom oosmHum> mo mHmmHmsm uouommlmso wan pom .Hoom umomv soHu nomHHou sumo Hound pmumnomuoosH Ho HHHoHHm «V smHmmo on» pH pom: sons moonuoz mmsmm upmHmEMm pom upome msu now sonHomsmEH usmnmomm mo mmoHpsH pmNHHmnmsmw .mw mHnme 90 so the loss of precision was due to the unequal cell fre- quencies of the post hoc methods. It clearly pays to incorporate the blocks into the experimental design prior to assigning units to treatments or use the analysis of covariance if its assumptions are satisfied. It should also be noted that the indices of impre- cision for the post hoc blocking methods were computed by averaging across all values of the harmonic mean in the emprical sampling distributions. If, in a particu- lar application, the cell frequencies should happen to be equal, the precision for equal cell frequencies would be essentially the same as for the a priori methods. If, on the other hand, the cell frequencies happen to be extremely disparate, then the precision is even less than that reported for this investigation. CHAPTER IV SUMMARY AND CONCLUSIONS The major purpose of the investigation of post hoc block formation methods was to investigate the feasibility of imposing a generalized randomized blocks design on obser- vations from a simple, one-factor design with fixed treat- ments when additional observations were available on some concomitant variable. The three post hoc blocking methods considered were post hoc blocking within treatments, fixed- range post hoc blocking, and sampled-range post hoc block- ing. For purposes of simplicity and comparability to earlier reseach, attention was restricted to the case of dependent and concomitant variables with homogeneous bivariate nor- mal distributions across treatment populations. For each post hoc blocking method, the major t0pics were: test- ing the hypothesis of null treatment effects under condi- tions of block-treatment additivity; the effect of intro- ducing non-additivity; testing the hypothesis of block- treatment additivity; and the precision of the method in the additive condition, relative to the one-factor ana- lyses of variance and covariance. The investigation relied heavily on Monte Carlo methods to generate empirical distributions of mean squares and 91 92 F ratios where analytic solutions were either unknown or unreasonably difficult. For each combination of treat- ments (t), blocks (b), average treatment-block cell fre- quency (n), and pOpulation correlation (:0 between the dependent variable and the concomitant variable, 1000 samples of observations were generated on the dependent variable (Y) and the concomitant variable (X). Within each sample, equal numbers of observations were randomly assigned to treatments and then assigned to blocks within each treatment group according to the three post hoc block- ing methods. A one-factor analysis of variance was com- puted on the unblocked observations, followed by unweighted means analyses for the three post hoc blocking methods. If any of the methods had zero cell frequencies, the sam— ple was dropped for that method and additional samples were later generated to bring the total number of samples to 1000. Empirical sampling distributions were generated for three separate conditions: (a) true null treatment effects hypothesis and no block treatment interaction (central treatments, additive condition); (b) false null treatment effects hypothesis and no block-treatment interaction (non-central treatments, addi- tive condition); (c) true null treatment effects hypothesis with block-treatment interaction introduced by generating 93 observations on X and Y such that one-half of the treat- ment populations had correlations equal to p and the remain- ing half had correlations equal to -p(central treatments, non-additive condition). Results of the investigation are summarized separately for each post hoc blocking method. Post Hoc Blocking Within Treatments The post hoc blocking within treatments method rank orders observations on X and forms b blocks, all of size n, within each treatment group. The observations on Y are then analyzed by a two-factor analysis of variance. This method has no effect on the variability of treat- ment means and affords no gains in precision. While the treatments mean square is identical to the treatments mean square for the one-factor design, the residual and interaction mean squares are reduced for o > 0.0 and neither the ratio MST/MS nor the ratio MST/MS has a R' TB central F distribution under the hypothesis of null treat- ment effects. Even if reference distributions were derived, fewer degrees of freedom would be available and the method would have less power than the one-factor analysis of variance. This method is specifically excluded from future remarks about post hoc blocking methods. 94 Fixed Range Post Hoc Blocking (FRPB) The fixed-range post hoc blocking method divides the pOpulation distribution of X into b ranges of equal proba- bility. The b-l quantiles of X determine the assignment of Y observations to blocks within treatment groups. An unweighted means analysis of variance is computed on the Y observations, subject to the restriction that all block- treatment combinations must contain at least one observa— tion for the method to be applicable. Empirical results supported the conclusion that blocks are a fixed factor and under the hypothesis of null treatment effects, the ratio MST/MSR has a central distribution, even under con- ditions of extreme non-additivity. The FRPB method had greater precision (and power) than the one-factor design when p > 0.6. Since the precision is partly a function of the harmonic mean of cell frequencies, the precision can be as great as for a priori blocking designs when cell freqeuncies happen to be equal or very poor if cell fre- quencies are extremely disparate. The ratio MSTB/MSR fol- lowed the central F distribution when block and treatment effects were additive and provided a reasonably powerful test for non-additivity, at least for the relatively extreme forms of non-additivity generated in this investigation. The conclusions about the FRPB method can be generalized 95 with some caution to clearly fixed blocking variables with prOperties other than those considered in the empirical investigation. Sex of the respondent or some other nomi- nal variable might be used to form blocks, provided that the usual analysis of variance assumptions are still met. Unless block effects are large or cell frequencies are very nearly equal, the FRPB does not provide a more power- ful test for non-central treatments than the one-factor design, but it does provide a means for investigating block- treatment interaction. Sampled-Range Post Hoc Blocking (SRPB) The sampled-range post hoc blocking method ranks obser- vations on X across all treatment groups, forming b blocks of size tn each. Blocks are formed within treatment groups according to the b-l sample quantiles of X and an unweighted means analysis is performed on the Y observations, provided that no block-treatment combinations have zero frequencies- Empirical results indicated that blocks behave like a finite effect where the treatments mean square contains an inter- action component, reduced by a factor of (1-b/B) where B denotes the unknown, but finite, number of blocks in the population. Nonetheless, the ratio MST/MSR had empirical Type-I error rates close to theoretical values for all but extreme cases of non-additivity, while the ratio MST/MSTB 96 was clearly too conservative for all non-additive cases considered. When block and treatment effects were addi— tive, the ratio MST/MSR followed the central F distribu- tion under the null treatment effects hypothesis and was consistently more powerful than the comparable test for the FRPB method when the null hypothesis was false. Like the FRPB method, the SRPB method was more precise than the one-factor design when p 3 0.6. The SRPB ratio MSTB/MSR had a central F distribution when block and treatment effects were additive and provided a reasonably powerful test for non-additivity when it was present. If the SRPB method is used to investigate non- additivity, e.g., forming blocks on a covariable when regres- sion heterogeneity is suspected, the additivity hypothesis should be tested prior to testing for treatment effects. If strong interaction is present, little faith can be placed in the test of the null treatment effects hypothesis, but then such main effects would have to be interpreted in the light of the interaction anyway. Like the FRPB method, the SRPB tests lose power as cell frequencies become more variable. When disparate cell sizes are encountered, precision may be increased ' by reducing the number of blocks until the arithmetic and harmonic means of cell frequencies are approximately equal. The increased residual variance should generally be offset by the larger harmonic mean and increased degrees of free- dom for estimating the residual variance. 97 If a reasonably strong correlation (p 3 0.4) exists between the dependent and concomitant variables and the assumptions are tenable, the analysis of covariance pro- vides a more powerful test of treatment effects than any of the other methods considered here, including a priori blocking methods (blocks are incorporated into the experi- mental design prior to assigning experimental units to treatment conditions). If the analysis of covariance is not applicable, a priori blocking methods are preferable to the post hoc blocking methods. The post hoc methods are more precise than the one-factor design only when block effects are very strong, but if non-additivity is suspected after data are gathered, either post hoc method provides a viable means for investigating the interaction. Under such conditions of non-additivity the SRPB test for treat- ment effects tends to be too liberal and should be used with caution--and a prior test for non-additivity. The a priori sampled range method, where blocks are formed on the basis of sample quantiles rather than popu- lation values, suffers from the same sensitivity to block- treatment interaction as the SRPB method. Contrary to Pingel's research (1968), the MSR is an apprOpriate error term for testing the null treatment effects hypothesis in the additive condition. However, the test becomes too liberal under conditions of severe non-additivity and should be preceded by a test for non—additivity with the ratio MSTB/MSR. BIBLIOGRAPHY ' BIBLIOGRAPHY Anderson, R.L. and T.A. Bancroft, Statistical Theory in Research. New York: McGraw-Hill, 1952. Bancroft, T.A., Topics in Intermediate Statistical Methods. Ames, Iowa: Iowa State U. Press, 1968. Cochran, W.G., Analysis of covariance: its nature and use. Biometrics, 1957, 13, 261-281. Collier, R.O., The effects of subject-treatment interac- tion in a randomized block, one factor experiment. Educational and Psychological Measurement, 1961, 21, Cornfield, J. and J.W. Tukey, Average values of mean squares in factorials. Annals of Mathematical Statistics, 1956, 21, 907—949. Cox, D.R., The use of a concomitant variable in selecting an experimental design. Biometrika, 1957, 44, 150-158. , Planning of Experiments. New York: Wiley, 1958. DeGracie, J.S., Analysis of covariance when the concomi— tant variable is measured with error. Dissertation Abstracts, 1969, 29, 3529B. Elashoff, J.D., Analysis of Covariance: a delicate instru- ment. American Educational Research Journal, 1969, 6, 383—401. Feldt, L.S., A comparison of the precision of three experi- mental designs employing a concomitant variable. Psycho- and M.W. Mahmoud, Power function charts for speci- fication of sample size in analysis of variance. Psycho- metrika, 1958, 23, 201-210. Fisher, R.A., The Design of Experiments. (8th ed.) Edin- burgh: Oliver and Boyd, 1935. 98 99 Fox, M., Charts of the power of the F-test. Annals of Mathematical Statistics, 1956, 21, 484-497. Glass, G.V. and J.C. Stanley, Statistical Methods in Edu- cation and Psychology. Englewood Cliffs: Prentice- Hall, 1970. Lindquist, E.G., Design and Analysis of Experiments in Psychology and Education. Boston: Houghton Mifflin, 1953. Lord, F.M., Large sample covariance analysis when the control variable is fallible. Journal of the American Statistical Association, 1960, 55, 307-321. Millman, J. and G.V. Glass, Rules of thumb for writing the ANOVA table. Journal of Educational Measurement, 1967' £1 41-510 Pearson, E.S. and H.O. Hartley, Charts of the power func- tion of the analysis of variance tests, derived from the non-central F-distribution. Biometrika, 38, 112— 130. Peckham, P.D., An investigation of the effects of non- homogeneity of regression slopes upon the F-test of analysis of covariance. Dissertation Abstracts, 1969, 22, 3531B. Pingel, L.A., A comparison of the effects of selected methods of block formation on the analysis of variance. Unpublished Ph.D. Dissertation, University of Wisconsin, 1968. Porter, A.C., The effects of using fallible covariables in the analysis of covariance. Unpublished Ph.D. Dis- sertation, University of Wisconsin, 1967. , How errors of measurement affect anova, regression analyses, ancova and factor analyses. Occasional Paper 13, Office of Research Consultation, College of Educa- tion, Michigan State University. and M. McSweeney, Randomized blocks design and non- parametric statistics. Paper presented at the American Educational Research Association Annual Meeting, March, 1970. Sarhan, A.E. and B.G. Greenberg (ed)., Contributions to Order Statistics. New York: Wiley, 1962. 100 Scheffe', H., The Analysis of Variance, New York: Wiley, 1959. Silverman, W. (adapted by R.G. Wolfe), Random floating point numbers in normal distribution (RANSS). Madison, Wisconsin: University of Wisconsin Computing Center, 1963. Snedecor, G.W. and W.G. Cochran, Statistical Meghods. Ames, Iowa: Iowa State University Press, 1967. Teichroew, D., Tables of expected value of order statis- tics for samples of size twenty and less from the nor- mal distribution. Annals of Mathematical Statistics, 1956, 21, 410-426. Wilk, M.B., The randomization analysis of a generalized randomized block design. Biometrika, 1955, 22, 70-79. and O. Kempthorne, Fixed, mixed, and random models. American_§tatistical Association Journal, 1955, 59, 1144-1167. and O. Kempthorne, Derived linear models and their use in the analysis of randomized experiments. WADC Technical Report No. 55—244, V. II, 1956. (1956a5 and O. Kempthorne, Some aspects of the analysis of factorial experiments in a completely randomized design. Annals of Mathematical Statistics, 1956, 27, 950-985. (1956b) "- APPENDIX A 101 Table A1. Summary Statistics for Ten Samples of One Thou- sand Pseudorandom Normal Deviates Generated by Subroutine RANSS 4_ :22 Range Variable Mean S.D. Skew Min Max 1 .081 .998 .041 -2.983 3.036 2 -.010 .991 -.095 -3.571 2.734 3 .017 1.000 -.091 -2.977 2.677 4 .027 1.008 .001 -3.320 3.201 5 -.004 1.024 -.051 -3.087 2.779 6 .029 .986 -.040 -3.435 2.937 7 .029 1.019 .039 -3.101 2.892 8 -1.00 1.021 .018 -3.419 3.485 9 -.040 1.002 -.051 -3.382 2.639 10 -.026 1.014 .019 -3.192 3.446 Combined .000 1.007 - -3.571 3.485 Median .007 1.005 -.012 -3.256 2.914 Correlations 2 3 4 5 6 7 8 1 1.000 2 -.048 1.000 3 .030 1.000 4 -.070 .068 -.021 1.000 5 -.004 -.056 .033 1.000 6 —.005 -.020 .021 -.026 -.015 1.000 7 -.032 .022 -.033 .023 .023 .030 1.000 8 -.012 -.044 -.017 .055 -.039 .007 1.000 9 -.032 -.026 .016 .045 .009 -.016 .008 10 -.002 .009 -.048 -.024 .009 -.009 .001 .041 Median r = .001 10 9 1.000 10 .021 1.000 APPENDIX B 102 Table Bl. Empirical Mean Squares for the Sampled-Range A Priori Blocking Method with Both Additive and Non-Addi- tive Block and Treatment Effects: 9 = 0.6; t = 2; b = 4; n = 5 Condition Source Additive Non—Additive Treatments .732 1.010 Blocks 4.661 .753 TxB Interaction .731 4.751 Residual .688 .693 103 Table B2. Observed Type-I Error Rates for the Sampled- Range A Priori Blocking Method with both Additive and Non- Additive Block and Treatment Effects: p = 0.6; t = 2; b = 4; n = 5 Error Rate (lOOOd) Ratio Dfl sz 750 500 250 100 50 10 MST/MSR (Additive) l 32 731 502 258 112 61 17 (Non-Additive) 1 32 782 574 338 157 98 40 MST/MSTB (Additive) 1 3 743 476 253 102 50 13 (Non-Additive) 1 3 444 132 14 O O 0 MSTB/MSR (Additive) 3 32 759 535 285 110 58 15 (Non-Additive) 3 32 999 991 987 955 916 775 Non-Additive Case constructed by: Yi = p*xi + /1-p2*Zi for treatment 1 Yi = -p*Xi + /1-p2*Zi for treatment 2 Ir—