ANALYSIS OF REPEATED MEASURES 7 DATA: A SIMULATION STUDY Elissertation for the Degree of Ph. :D, MICHIGAN STATE UNIVERSSTY VERDA M. SCHEIFLEY ' 1974’ ....~. ‘ ”mull "Illgllljlgl lllllllljlgll ‘1 10666 51 “ : This is to certifyrfh‘at the thesis entitled - w ANALYSIS-OF RE?EATED MEASURES DATA: A SIMULATION STUDY presented by Verda M.,Scheifley has been accepted towards fulfillment of the requirements for _Bh.D_degree 1n flounseling, Personnel ServiCes, and Educational Psychology wtglfl VA 35; We: Major professor 0-1639 OVERDUE FINES ARE 25¢ PER DAY PER ITEM Return to book drop to remove this checkout from your record. ABSTRACT ANALYSIS OF REPEATED MEASURES DATA: A SIMULATION STUDY BY Verda M. Scheifley An experimental design frequently encountered in the behavioral sciences is one in which subjects are measured on the same variable at different points in time or under dif- ferent experimental conditions. One interest of the exper- imenter involves contrasts among these repeated measures. This study proposes three statistical analyses which are appropriate to repeated measures data--classical mixed model analysis of variance (ANOVA), multivariate analysis of vari- ance of repeated measures (MANOVA of RM), and analysis of covariance structures (ANCOVST). In ANCOVST the additional question concerning the sources of variation in the subjects' performance over time or across conditions can also be explored. These three models differ in their assumptions con- cerning the correlation between the latent random variables underlying the repeated measures. When using mixed model ANOVA, one assumes that the latent variables are not corre- lated. MANOVA of RM specifies a general positive definite Verda M. Scheifley correlation matrix for the latent variables. .ANCOVST accounts for correlation among the random components but also enables one to fit a more parsimonious model with fewer latent compo- nents. The focus of the study is the effect on the sampling distributions of the test statistics and parameter estimates for each of the three repeated measures models when applied to different types of data some of which conform to the assumptions of the procedures, others of which do not. More specifically, probability of Type I error, power of the test, biasedness of parameter estimates, and the relative effi- ciency of the estimates are examined for each model. The method of exploring these issues involve the use of simu- lated data which are generated by computer algorithms and enable one to investigate the properties of statistics by observing their distributions over a large number of samples where the pepulation parameters are known. The data generated for the study are from a mul- tivariate normal distribution with a mean vector E and a covariance matrix 2. There is one group of subjects, i.e. a single sample, with a 22 factorial arrangement over the repeated measures. Three population covariance matrices were chosen such that each was appropriate to the assump- tions of one of the three analysis routines. Three popu- lation mean vectors were chosen such that there were no repeated measures effects, there were slight effects, and Verda M. Scheifley there were large effects. The choices of E and 2 form a 32 'crossed design of possible combinations of population parameters. One thousand samples of thirty subjects each were generated from the nine populations. Each group of samples was analyzed by all three routines and estimates of a, the power, and repeated measures parameters were found. When the assumptions of mixed model ANOVA are met and there are no repeated measures effects, MANOVA of RM and ANCOVST are more likely to reject the null hypothesis than mixed model ANOVA at the .05 level. The rate of rejection is similar at the .01 level for all three models. When 2 violates the mixed model assumptions, all three tests are generally conservative for this situation. ANCOVST is always more powerful in detecting slight effects than the other two procedures. When assumptions are met, mixed model ANOVA is the more powerful of the remaining procedures. When the size of the effects is increased, MANOVA of RM is always most powerful. In this situation, ANOVA is more powerful than ANCOVST when the assumptions are met. To investigate biasedness and efficiency, the repeated measures effect and standard errors were estimated by two methods. Mixed model ANOVA and MANOVA of RM used least squares estimation. Maximum likelihood estimates are found in ANCOVST. The two estimation procedures give essen- tially identical estimates for the repeated measures effects. Verda M. Scheifley One property of least squares is unbiasedness which is supported by the data in that the estimates are very close to the population parameters. Since ANCOVST estimates of effects are the same, maximum likelihood estimates in this situation are also unbiased. The standard error terms found in ANCOVST are generally smaller than for the least square estimates implying that maximum likelihood estimators are slightly more efficient than those of least squares. ANCOVST also finds estimates of the covariance components of the model by maximum likelihood procedures. The empir- ical estimates are different than the population parameters, however, suggesting a degree of biasedness in these estimations. ANALYSIS OF REPEATED MEASURES DATA: A SIMULATION STUDY BY Verda Mfischeifley A DISSERTATION Submitted to Michigan State University in partial fulfillment of the requirements for the degree of DOCTOR OF PHILOSOPHY Department of Counseling, Personnel Services, and Educational Psychology 1974 ACKNOWLEDGMENTS I would like to take this opportunity to thank the many peOple who made it possible for me to attain a doctoral degree. First and foremost, I acknowledge the members of my guidance committee who encouraged and supported me at every point of my graduate studies. Dr. William Schmidt served as my advisor, committee chairman, counselor and friend. His patience and understanding were especially appreciated. Working with him has greatly increased my knowledge and, more importantly, my interest in research and teaching. Drs. Maryellen McSweeney, Andrew Porter, and Robert Staudte contributed greatly to my dissertation and to my profes— sional development. I am grateful for this assistance. I would also like to take this opportunity to acknowledge the support I received as a National Defense Act Title IV fellow. Special thanks go to David Roderick and other com- puter center employees for their cooperation and assistance in the many computer problems which were part of this study. I would also like to thank Myrna Russell and Shirley Wagner for typing the rough draft of this document. And finally I would like to thank my parents and friends for their continual support and pep talks while I worked on my dissertation. ii LIST OF Chapter I. II. III. IV. VI. APPENDIX TABLE OF CONTENTS TABLE S O O O O O O O O O 0 O O O O 0 O 0 STATEMENT OF THE PROBLEM . . . . . . . . THREE METHODS OF ANALYSIS FOR REPEATED WASUM S DATA 0 O C O 0 O O O O O O O O O Univariate Analysis of Variance . . . . Multivariate Analysis of Variance of Repeated Measures . . . . . . . . . . Analysis of Covariance Structures . . . REVIEW OF THE LITERATURE . . . . . . . . Simulation Studies of Repeated Measures Designs . . . . . . . . . . . . . . . Data Examples of Repeated Measures Designs . . . . . . . . . . . . . . . The Present Research . . . . . . . . . SIMULATION PROCEDURES . . . . o . . . . . Description of Population Parameters . Description of the Generation Routine . Test of the Fit of the Generators . . . Analysis Routines . . . . . . . . . . . Measures of Type I and Type II Error, Biasedness, and Efficiency . . . . . SIMULATION RESULTS . o . o . o o o o o . Results for the Hypothesis Tests . . . Results for Estimation Phase . . . . . IMPLICATIONS OF RESULTS . . o o o . . . . Guidelines for the Behavioral Scientist Further Research . . . . . . . . . . o O O O O O O O 0 0 O 0 O 0 0 O 0 J C I O BIBLIOGMPHY o O O O 0 O O 0 O 0 0 O 0 0 o 0 O 0 iii Page iv 13 22 34 36 ~42 45 46 48 51 57 6O 62 68 69 79 89 89 94 96 101 LIST OF TABLES ANOVA Table for Univariate Mixed Model Analysis of Variance for Repeated Measures . . . . . . . F Ratios and Degrees of Freedom Mixed Model ANOVA O O O O O O O O I O O O O O O O 0 O O O o Multivariate Analysis of Variance of Reparameterized Repeated Measures (One- Sample Case) 0 O O C O O O C O O O O 0 e O 0 0 Composition of Sum of Square Matrices from Table 2-3 for Given Example . . . . . . . . . . Structure of T and W . . . . . . . . . . . . . Partial Derivatives of the Log Likelihood Function 0 O 0 o O O O O I O O O O O O O o o 0 General Structure of u and Z 32 Design of Populations . . . . . . . . . . . . . . . . . . Population Covariance Matrices . . . . . . . . Population Mean Vectors and Power . . . . . . . Test of Fit for Sample Covariance Matrix for N = 30 and K = 4 I O O O O O 0 O 0 O O G O 0 Estimates of P(Type I Error) and Power for the Three Analysis Routines and Their Standard Error Terms . . . . . . . . . . . . . P(Type I Error) and Power Estimates Unique to Mixed MOde 1 ANOVA O O O O 0 O O O O O O O O P(Type I Error) and Power for the Three Analysis Routines Asymptotic Chi Square Test 0 O O 0 O 0 O O O O O O O O O O 0 O O O 0 iv Page 10 l3 19 19 28 31 50 52 53 60 70 74 76 TABLE Page 5-4 Test of Fit for ANCOVST Model . . . . . . . . . . 79 5-5 Least Square Estimates and Standard Errors MANOVA of RM and Mixed Model ANOVA . . . . . . . 81 5-6 Maximum Likelihood Estimates and Standard Errors for Eé ANCOVST O C O O O O O O C O O O O O 82 5-7 Maximum Likelihood Estimates and Standard Errors for Tl/Z and W1 . . . . . . . . . . . . . 86 1/2 1/2 5-8 Population Values of T and W . . . . . . . 87 5-9 Maximum Likelihood Estimates and Standard Errors for T and W . . . . . . . . . . . . . . . 88 CHAPTER I STATEMENT OF THE PROBLEM An experimental design frequently encountered in the behavioral sciences is one in which subjects are measured on the same variable at different points in time or under dif- ferent experimental conditions. In such situations, one interest of the experimenter involves contrasts among these repeated measures. For example, a group of students are measured on their reading comprehension at the beginning and at the end of a reading improvement program. At each time point they are measured on the same reading comprehension test but given over two types of reading materials such as fiction and nonfiction. The four measures can be conceived ' of having been generated from a 2 x 2 factorial design with a pre-post dimension and a fiction-nonfiction dimension. Concerns of the researcher are differences in reading comprehension between the two time points, differences in comprehension between the two classifications of reading material and the interaction of these two dimensions. Data of this kind require an analysis procedure which can answer all these research questions. Lana and Lubin (1963) reported a study to discover the frequency of repeated measures designs and how they were analyzed in psychology. Three journals (Journal of Experi- mental Psychology, Journal of Physiological and Comparative Psychology, and Journal of Abnormal and Social Psychology) were examined for the years 1957-1959 to determine the kinds of analyses used. About half of the articles (593) used the analysis of variance to investigate the hypotheses of inter- est. Of these articles, almost 40% had a repeated measures design (20% of all articles). Although this is a small sub- set of research, it does show that the repeated measures design is used quite frequently, at least in psychology. Another point mentioned in the article was the method used to analyze the data. The majority used a uni- variate two-way analysis of variance procedure (subjects x repeated measures) or essentially the classical mixed model ANOVA. This is appropriate in that the question of interest can be explored by these procedures, but it is often inap- propriate for empirical data due to violation of the assump- tion that the latent random components of this model are uncorrelated. This shortcoming of the mixed model led to the development of other procedures to answer the same question with no such restrictions. One procedure evolved from the approach of multivariate analysis of variance and allows for correlation between the latent variables. This is termed the multivariate analysis of variance of repeated measures. Another procedure, analysis of covariance structures, not only accounts for correlation among the random components but also enables one to fit a more parsimonious model with fewer latent components. In this approach the additional question concerning the sources of variation in the subject's performance over time or across conditions can also be explored. It has been noted in the literature for the classical mixed model ANOVA that the assumption concerning the inde- pendence of the random components of the model is both a critical and a nebulous one (Scheffe, 1956). Thus, other options available to a behavioral scientist, such as the two other procedures mentioned above, should be explored as pos- sible alternatives. The question of which approach to apply in a given repeated measures situation and the effect of applying each to the same situation is of interest to researchers. The focus of the present study is the effect on the sampling distribution of the test statistics and parameter estimates for each of the three repeated measures analyses when they are applied to different types of data. More specifically, the three analyses will be performed on repeated measures data generated from populations with one of three covariance matrices. Each covariance matrix will be designed so that it is appropriate to one of the specific analysis procedures. For example, one matrix is derived from data which meet the assumptions of the mixed model ANOVA. Of interest are the results for each of the analyses when performed on data which violate the assumptions of that model, i.e., data generated from a population in which the covariance matrix does not meet the requirements. It is important to the researcher to know if the probabilities of committing a Type I and Type II error change or if the standard errors of estimates of contrasts change as the pro- cedure is applied to a covariance matrix which does not meet the assumptions of that model. The general purpose of this dissertation is to dis- cuss each of these three methods of analysis appropriate for the repeated measures data and their assumptions and to give guidelines based on empirical simulation for use when these assumptions are violated. The method of exploring these issues will involve the use of simulated data which are generated by computer algorithms and enable one to investi- gate the properties of statistics by observing their distri- butions over a large number of samples where the population parameters are known. By analyzing a large number of sam- ples one can compare (1) the resulting empirical distribu— tion of the test statistics for each analysis method to the theoretical distribution for that model and (2) the empir- ical standard errors for parameter estimates. These results can be used to help determine the appropriateness of each of the analyses to different data situations. CHAPTER II THREE METHODS OF ANALYSIS FOR REPEATED MEASURES DATA A general model which can be used for single sample repeated measures data is the mixed model where the subjects are considered as a random factor and the repeated measures as a fixed dimension. The parameters of the model include the general mean, fixed effects for repeated measures, a random effect for subjects, random effects for the inter- action of subjects and repeated measures, and a random error term. When the design over the repeated measures is a 2 x 2 factorial, one model generally used for the score of the ith subject on each occasion (repeated measure, occasion, and trial will be used interchangeably) is (2-1) yijk = u-taj-+8k-ta8jk-+ci-+caij-+c8ik-+ca81jk-+eijk where u is the grand mean, aj is a fixed effect due to the jth level of the first factor of the repeated measures design, 8k is the fixed effects due to the kth level of the second factor of the repeated measures design, aBjk is the effect due to the interaction of the factors in the repeated measures design, ci is the random effect due to the ith ‘ subject, caij is the random effect due to the interaction of the ith person with the jth level of the first factor, CBik is the random effect due to the interaction of the ith per- son with the kth level of the second factor, caBijk is the random effect due to the interaction of the ith person with the jth level of the first factor and the kth level of the second factor, and eijk is the random error term. The data matrix appropriate to such a situation is presented in Figure 2-1. a a a 1. 2 p b1 b2... bq b1 b2... b . . . b1 b2... b SI 82 Sn Figure 2-1. Data matrix for a two-way factorial design over the repeated measures. Repeated measures data can occur in ways other than the one represented in Figure 2-1. The design over the repeated measures can involve only one factor or it can involve more than two factors and the subjects can also be stratified into various subgroups. The present discussion, however, will be limited to the single sample case with four repeated measures having arisen from a 22 factorial arrangement (see Figure 2-2). Figure 2-2. Data matrix for a 22 design over the repeated measures. Research hypotheses may be concerned with both the fixed and the random components of this model. Three analytical procedures are proposed to test such hypotheses. They are: l. Univariate mixed model analysis of variance 2. Multivariate analysis of variance of repeated measures 3. Analysis of covariance structures. Each analytical method, its assumptions, the hypotheses for which it can be used, and the appropriate test for each hypothesis of interest will now be discussed. Univariate Analysis of Variance The classical method of analysis for the model in (2-l) is the mixed model analysis of variance (ANOVA). The assumptions of this model are that the subjects are sampled from a population in which 1. c is distributed N(O, 0:) 2. caj are distributed N(O, odd) 3. ask are distributed N(O, 0:8) 4. caBjk are distributed N(O, oéas) O 0 2 5. ejk are distributed N(O, Ge) 6. All random components, c, caj, CBk’ caBjk, and ejk are considered to be pairwise independent random variables implying the covariances between the latent variables are all zero. The restrictions placed on the model by these assumptions, especially assumption 6 concerning the correlation between the random components, are important and need to be consid- ered when employing mixed model ANOVA for repeated measures data. In this context, one set of questions of interest to the experimenter is whether or not there are non-zero design effects over the repeated measures. The null hypothesis that there are no effects due to the repeated measures can be stated as: a1=a2=...=ap=o (2-2) HO: 31=32= ...=Bq=o a811=0812* ...=a8 =0 Data which are described by the model in (2-1) and for which the foregoing assumptions hold, can be analyZed using the mixed model ANOVA. The total sums of squares for observations can be divided into four main parts-—the sums of squares for the constant term or grand mean, ssm, the sums of squares for the subjects, ssc, the sums of squares for the repeated measures effect, ssrm, and the sums of squares for the error term, sse. It is common to further subdivide the sums of squares for repeated measures and for error to reflect the factorial design over the repeated measures. In this case, ssrm is partitioned into three parts of interest--a term for the first factor in the design, ssrma, a term for the second factor in the design, ssrmb, and a term for the interaction of the two factors, ssrmab. The sums of squares for error is also partitioned in a similar fashion--sse sseb, sseab. Formulas for the a’ sums of squares, the corresponding degrees of freedom and expected sums of squares are given in Table 2-1 (see Winer, 1962). The tests for the hypotheses of interest as given in (2-2) employ F ratios calculated from the appropriate sums of squares values and df from the ANOVA table (Table 2-1). Before testing the main effects for occasions one must test for an interaction between factors A and B. The F ratio is 10 n xa xn a na x xna n a o “H o o h o o H e a “51363 so 30 xx. saw: a: 80 mixed“ 5 EVEN E x nd a Mn xa n xnfl fl . . h o . n . a n . e0 Phenom“ E .5 WEEKS 5 e3. Asians: 5 e95 missus E .mmm .m .mmmH .HocHS mom .3oHon oocwmoo mum monsoon mo meow Hom menses “av-Amv+dvv+fimv+. no mewe+ we STE-3T5 u «we 3-3 2-8 2-5 as :ofleeuoufi ewes.r we Sis-37:: - new... 2-3 3-5 m H88... ewe? we 31812-5 - new... are 3-5 a .883 HON-Hum awed... mowo+ we ISIS-ELM: u Assume 3-3 2-8 m2 cofluomuoucH woms+.mwoml.wo Aavlfiev u naumm and m Houomm weee+ewee+ we 3-5 - «8% Te a Seed monummoz woumomom wee? we mic-3e - 8e Ta 383% ...mc u 8mm H acoumcoo mwumswm mo moumsvm mo meow Eooomnm downwaum> mo monsom mfism concomxm mo mooumoa mousmmoz voucomom Mom moccaum> mo mammHmsm Home: vmxwz oumflum>ficn How THAMB ¢>Oz¢ HIN mqmda ll ssrmab/(p-l)(q-1) (2'3) = sseab/(n-1>(q-1) 1"“ 1’ Multivariate Analysis of Variance of RepeatediMeasures To avoid making the assumption that the latent random variables of the model in (2-1) are uncorrelated, a multivariate analysis of variance (MANOVA) has been sug- gested to analyze repeated measures data. This analysis is not really appropriate for the situation as there is no test for differences across the repeated measures dimension. Bock (1963) and Potthoff and Roy (1964), however, have sug- gested a variation of MANOVA which does include a repeated 14 measures test. This method will be referred to as the multivariate analysis of variance of repeated measures (MANOVA of RM). The model in (2-l) can be rewritten in matrix terms as _ * (2‘6) 11 " Ul+l+cli+gi+ii where xi is a k x 1 vector of observations for the ith subject, 1.is the k x 1 vector of fixed effects for the repeated measures, cli is the random effect for the ith subject, and g; is the vector of random effects for the subject by repeated measures interactions, and 3i is the k x 1 vector of error terms for each of the k repeated measures. In a two way design the number of occasions, k, is equal to the product of the number of levels for each factor, e.g., k = pq. Substituting 1=ui+x and * 8. = - + .. + . -1 Cl; 21 21 the model in (2-6) can be rewritten as a general multi- variate model (2-7) Xi = l + e where 1.is the k x 1 vector of means for the repeated measures and g’is the k x 1 vector of sampling errors 15 distributed N(Q,£) where Z is any symmetric positive definite matrix. Note that this general formulation of the model can be collapsed to the mixed model if such assumptions are appropriate. Characterizing the structure of the vector l’is generally one of the main concerns of the experimenter both in estimation and in hypothesis testing. One null hypothesis, for example, might be (2-8) H : T = T = ... = The structure of l_for repeated measures data having arisen from a factorial arrangement can be characterized by a linear model which includes parameters for the main effects and the interaction effects. This can be written as: (2-9) I A g kxl kxm mxl which implies (2-10) Xi = Ag + Si where A is the design matrix appropriate to the factorial arrangement over the repeated measures and g’is the vector of parameters characterizing the treatment effects. Con- sider the single-sample case with 22 design over the occa- sions. In this context A and g are described as follows: l6 .1 A Q '11” "110101000” "- ' (2_11) r2 = 1 1 0 0 1 0 1 0 0 a1 13 101100010 012 5% L1°1°1°°°11 81 B2 a181 6182 (1281 Lazgfl The design matrix A is not of full rank and so the param- eters of g are not identifiable. The model can be reparam- eterized which is equivalent to factoring A into the product of two matrices, K and L (Bock, 1972), A = K L kxm kxfi fixm where L describes a set of k linear combinations of g in which the experimenter is interested and K is the corre- sponding column basis for the design matrix A. One possible choice for K in this example is an orthogonal matrix (2-12) K = H-Ahap HPHFJH HrALud HF4;4H The implied L is 17 1 1/2 1/2 l/2 1/2 l/4 1/4 1/4 1/4 0 1/2 -1/2 0 O 1/4 1/4 -1/4 -l/4 O O O 1/2 -1/2 1/4 -l/4 1/4 -l/4 0 O O 0 0 1/4 -1/4 -l/4 1/4 The new expression for (2-9) under this reparameterization is (2-13) I = KL; = KO where-Q = Lg is a k x 1 vector describing the resulting linear combinations. The first element of g characterizes the constant term which is equal to the grand mean when there are no repeated measures effects. The second and third elements characterize the contrasts between the two levels of factor A and of factor B, respectively, and the fourth element represents the interaction contrast. Substituting the results of (2-l3) into (2-10), the vector of observations is given by: (2-14) yi = Kg + _e_i. If Xi is transformed by premultiplication by K-1, -1 -1 (2-15) K xi 1 K Kg+K _e_i where g: is distributed mg, 2* = (K’1)’2K). The k x 1 vector gbis then the vector of population means under the transformation or contrasts among the repeated measures and can be estimated by the transformed sample mean 18 (2-16) 9 = K x. The last three contrasts in Q represent different effects for the repeated measures. Thus, a test for the signifi- cant difference of these effects from zero is equivalent to the test of the null hypothesis in (2-8), i.e., T1 = T2 = ... = Tk. The hypothesis testing stage in the MANOVA of RM is parallel to that of the mixed model ANOVA. In both analyses, concern is with the effects due to the repeated measures. In the mixed model, the null hypothesis is tested by divid— ing the total sums of squares term into appropriate parts as described in Table 2-1 and forming the appropriate F ratios. The total sums of squares term is now replaced by a matrix of sums of squares and cross products, SST* in the multi- variate analysis. In MANOVA of RM, the SST* matrix is similarly partitioned into a between and a within part, SSM* and SSE* which are used to form multivariate F ratios for hypothesis testing. The relationship between the sums of squares in the univariate analysis and the elements of these matrices will be discussed later. The MANOVA table for the transformed repeated measures analysis is given in Table 2-3. The structure of SSM* and SSE* is illustrated in Table 2-4. 19 TABLE 2'3 Multivariate Analysis of Variance of Reparameterized Repeated Measures (One-Sample Case) Source of Sum of squares Expected sum of variation df and products (kxk) squares and products Constant l SSM* = nK'yY'K K'(Z+nTT')K subjeCts n-l SSE* = K'(Y'Y-nyy')x (n—1)K'Zx within group Total SST* = K'Y'YK Composition of Sum of Square Matrices from Table TABLE 2-4 2-3 for Given Example SSM* = SSE* = ssae L ssm ssmrm* a SS .3. ssmrm* ssa ssae ssae g.» 0‘» m » ssrm ssrm figfifi-D’fl- ssrm 2b W 586 536 g.* m * sse 2b m & ssrmb * ssrm ab2 sse * U‘» sse * ssrm ab * SSE Constant term a -a effect 1 2 Bl--B2 effect Interaction effect Subjects within group Subject x al-a2 error 8 error Subject x B 2 1 Subject x interaction error 20 When MANOVA of RM is used to analyze repeated measures data, the previous assumption of uncorrelated latent random components in the model (2-1) is no longer necessary. The fact that there are non-zero off diagonal elements in SSE* illustrates why this is true. The off- diagonal elements divided by the appropriate degrees of freedom estimate the covariances in 2* which would be zero if the foregoing assumptions were valid. The use of multi- variate tests of significance in conjunction with these procedures then takes into account any correlation between the latent components. One does not need to adjust the degrees of freedom as with the mixed model ANOVA when the assumption of uncorrelated random components was violated. As in the mixed model ANOVA, an F-ratio is now formed to test the null hypothesis in (2-8). In the clas- sical ANOVA procedures the F was a ratio of mean squares but in MANOVA of RM the F ratio is found from the latent roots of SSM* in the metric of SSE*° The method involves the partitioning of SSM* and SSE* as shown in Table 2-4 and extracting the submatrices which hold the effects of inter- est. In the example, the 3 x 3 submatrices in the lower right hand corner of SSM* and SSE*, referred to as SSMk-l and SSER-l’ respectively, represent the repeated measures effects with which the null hypothesis is concerned. Solving the determinantal equation 21 * _ * = (2 17) |ssmk_l Asssk_1| o for the k-l characteristic roots of SSM]:_l in the metric * one of the multivariate test criteria can then k-l' be applied. One such test is the F statistic due to Rao of SSE which is based on Wilks A and is given by: where A1 is the ith characteristic root of (2-17). Under the null hypothesis, the test statistic F is given by (2-18) F = l-Al/t . mt-ZS Al7t knh where m = ne + nh - (nh+k+l)/2 _ 2 2_ 2 2_ t — [(k nh 4)/(k +nh 5)] 8- (11h p-2)/4 and has a univariate central F distribution with nhk and mt-ZS degrees of freedom. In (2-18), ne is the degrees of freedom for error, nh is the degrees of freedom for the hypothesis, and k is the number of dependent variables. A significant result would suggest a repeated measures effect. It is also possible to test for the presence of each of the three effects analogous to the mixed model ANOVA by using step-down F testing procedures (Bock, 1969). This approach is not followed in the present study. 22 All the sums of squares needed to perform the mixed model ANOVA can be found in SSM* and SSE* for MANOVA of RM. The diagonal elements of SSE* and SSM* (Table 2-4) are equal to the sums of squares for repeated measures in Table 2-1. Thus, if one begins an analysis of repeated measures data with the procedures of MANOVA of RM and discovers that the assumptions of mixed model ANOVA are met, the information is available for this analysis and it is not necessary to begin again. Statistical tests of whether or not the data meet the assumptions of the univariate model for certain applications are available (see Winer, 1962; and Kirk, 1968). The calculation of F ratios for the mixed model ANOVA pro- ceeds as defined by (2-3), (2-4), and (2-5). Analysis of Covariance Structures A third model which can be used for the analysis of repeated measures data is the analysis of covariance struc- tures (Bock and Bargman, 1966; Jareskog, 1970; Wiley, Schmidt, and Bramble, 1973; and Schmidt and Wiley, 1971). The data situation remains the same. Subjects are measured on the same variables at different points in time or under different experimental conditions. The question of differ- ences across the repeated measures still applies but an additional question can also be explored, i.e., what con- clusions can be drawn about the sources of variation in the subjects' performance? 23 This model has an advantage to the mixed model ANOVA in that it does not require the restriction of assumption 6, i.e., it allows for correlated latent random components. Analysis of covariance structures (ANCOVST) also has an advantage over MANOVA of RM in that a more parsimonious model with fewer parameters can be fitted to the data than the general model assumed in MANOVA of RM. For example, one can specify certain random components to have zero means and zero variances and fit a more restricted model with fewer parameters. If the model fits this should result in greater efficiency for the ANCOVST procedures when compared to the MANOVA of RM procedures. Rearranging and grouping the terms in (2-1), the model can be rewritten as (2-19) yijk ,= (n+ci) + (aj+caij) + (Bk+c8ik) + (aBjk+caBijk) + eijk where each set of terms in parentheses has both a fixed effect and a random component. For example, for the first set, (u+ci), u is a fixed effect and ci is a random compo- nent. Its expected value under the assumptions of the mixed model is given by E (u+ci) E01) + E (Ci) n+0 =11 24 and its variance can be written as Var(u+ci) Var(u) + Var(ci) Similarly, the expected values for the other groups are d, and a res ectivel with variances of 02 02 and 02 . 8! Bl p Y! C0, C8! C08 An alternative specification of this model which is the one used for ANCOVST would be ~ =c.+a.+b +eb. +e- (2-20) ykjk 1 l] ik ijk ijk where ci, aij’ bik' abijk and eijk are all random variables . - 2 2 2 With means u, dj, Bk, aBjk, 0 and variances oc, oaj, obk, Uéb'k' 0;, respectively. The model as specified in (2-20) 3 compares with that in (2-18) in that Ci = (u+ci) aij = (aj+caij) bik = (Bk+CBik) abijk = (aBjk+CaBijk) In order to estimate and test the effects of inter- est, the model in (2-20) can be reparameterized as in MANOVA of RM. Formulating the model in matrix terms as in (2-9), the ANCOVST model can be rewritten as 25 (2-21) Xi = Agi + £1 where l l 0 l 0 A = l l 0 0 l l 0 l l 0 l '0 l 0 l and F- - c a1 51 = a2 b 1 b2 _ 1 1 Note that this formulation assumes a main class model for the 22 example which is an advantage of ANCOVST. As in MANOVA of RM, the design matrix A can be factored into two matrices K and L where a possible selection for K and L is l 1 1 _ l 1 -1 K“ 1 -1 1 1 -l -1 and 1 1/2 1/2 1/2 1/2 L = 0 1/2 -l/2 0 0 0 O ' O 1/2 -l/2 Thus (2-22) 11 = KL 531 + _l =K (5. + e 26 where éi = Lgi. The vector éi is the random vector of contrasts specified for the model. The expected value of 2 describes the contrasts between the fixed effects of the model in (2-18) such that (2-23) E(y_) = p = KE(_~O_i) = REE) where P j d1+02 81+82 u-+ Grand mean 2 2 pa = 01 - a2 Contrast for Factor A B1 - 82 Contrast for Factor B it. _ To simplify notation, each element in Eé will be referred to as “Oi where i=1, 2, 3. The estimate of E(§i) is found from 1 - (2-24) E(9_i) = 35 = (K'K)— K'y_. The variance of the observations can be expressed as (2-25) Var(y) = Z = K¢K' + T where T is the covariance matrix of the random vector Q and W is the diagonal covariance matrix of g. The estimation of E5! 0, and W in this model enables one to look at the sources of variation among the subjects and the effects across the repeated measures instead of just the latter as in the other two models. 27 The diagonal elements of 3 provide estimates of the variance components associated with the random effects of E and the off diagonal elements of 3 give the estimates of the covariances between these random effects. The elements in P, a diagonal matrix, provide estimates of the error vari- ances. The vector fié gives estimates of the specified contrasts or effects over the repeated measures. The struc- ture of the covariance matrix of the observed variables, 2, depends on the assumed structure for 0 and T. For example, when the random components are assumed to be uncorrelated, Q is specified as a diagonal matrix. This is called the orthogonal case. When the error variances are assumed equal, 9 = 021 is a diagonal matrix with all elements on the diago- nal equal. This is called the homogeneous case. When T is orthogonal and T is homogeneous, the assumptions for the mixed model ANOVA are met. The latent'random components are often correlated such that T is no longer a diagonal matrix but ¢ is a general symmetric positive-definite matrix. In this situ- ation, T is said to be oblique. It is also the case that the error variances are not always equal such that the diag- onal elements of W are unequal instead of equal as in the homogeneous case._ In this situation 9 is said to be heter- ogeneous. For the given 22 example, the alternative structures for T and W are characterized in Table 2-5. 28 TABLE 2-5 Structure of 9 and W To C)» 9 Orthogonal w Homogeneous 7 2 (symmetric) Foe (symmetric T 2 Oé 0 U: 2 2 2 O 003 O 0 Ge J 2 _O O 0 0e J ¢ Oblique H’Heterogeneous (symmetric) o: (symmetric) 1 2 2 O 0 O 2 62 e2 2 g 2 CG CG 0 0 Ce 3 2 3 3 3 .J 0 0 0 0: L “.1 29 Since one of the advantages of ANCOVST is the ability to specify a more parsimonious model than a general structure of Z, T is described for the main class model only--omitting the AB interaction term of the 22 example. Combinations of the restrictions for T and 9 form four specifications for 2. These four cases are summarized as follows: Case Specification for T Specification for W A Orthogonal Homogeneous B Orthogonal Heterogeneous C Oblique Homogeneous D Oblique Heterogeneous The main focus of the present study in ANCOVST is Case A and C concerning the two degrees of restrictions placed upon T while W is assumed homogeneous. The experimenter can form two hypotheses in the ANCOVST model. One hypothesis concerns the Specification for the structure of 2. For example, it may be reasonable to assume the random components to be correlated and the error components homogeneous (Case C). The null hypothesis is that the model of Case C fits the data. Rejection of the null hypothesis implies that the model does not fit or that Case C restrictions are not appropriate for the given set of data. The second hypothesis concerns the repeated measures effects of 36' As with mixed model ANOVA and MANOVA of RM, the null hypothesis is that there are no repeated measures effects. This is equivalent to setting pa and “6 to 2 3 30 zero. Rejection of Ho implies there is an effect over the occasion. The principle of maximum likelihood is employed to estimate Eér T, and 9. For this method the estimates of the parameters are chosen so as to maximize the likelihood func- tion which is the joint density of the observations. (2-26) L(p-£) 1 M; exp {—1/2 (xi—y'z'lmi-gn. (2n)anZI i IIMS l The values which maximize a function also maximize monotonic functions of it and so for convenience one frequently works with the natural logarithm of the likelihood in (2-26). (2-27) 2(B,z) = -1/2 nk£n(2n)-1/2 nianl n -l -l/2 21(yi-E)'Z (Xi-p). 1: The interest here is not in estimating E and 2 per se but in estimating Eé' T, and 9 which define the structure of B_and 2 (2-23 and 2-25). Substituting from (2-23) and (2-25) into (2-27) the log likelihood function is (2-28) 2(Eé,¢,w) = -1/2 nk2n(2n)-1/2 nznlxex'+w| n ~ -1 —1/22 (xi-Kg) ' (KTK'+‘P) (xi-Kg) . i l 31 To find the maximum of the likelihood function, one must find the values for which the partial derivatives of (2-28) with respect to 38' Tl/z, and 91/2 are zero, or az‘flél¢lw) ag<£él¢lw) 31(Eél¢lw) = 0 = 0 = 0 The expressions for the partial derivatives are given in Table 2-6. The derivatives are expressed in terms of T1/2 1/2 and V which are the cholesky factors of T and 9, respectively. The cholesky factor of a matrix is analogous to the square root of a scalar. The cholesky factor of T, 1/2 T , is a lower triangular matrix with zeros in the upper 1/2 ¢1/2' half of the matrix such that T = T. TABLE 2-6 Partial Derivatives of the Log Likelihood Function 31"” = -N diag{T1/2K'$2K}a ———31“/2 = -N(1/ Zx'mo 3T (orthogonal) 3T (oblique) 312/2 = -N 0 arm} -—§1&;§ = -N diang/ZSZ} 8W (homogeneous) 3W (heterogeneous) §§E-=i-N(xvz‘1(2-20)) Be a0 = 2'1 - z'lsz'1 - flog-E) (3-11) 'z'l. 32 1/2, is a diagonal matrix with The cholesky matrix of T, W elements equal to the square root of the diagonal elements of W. The estimates of the parameter matrices are found by solving the equations by means of the numerical iterative technique of Fletcher and Powell (1963) as they cannot be solved directly. Hypothesis testing for ANCOVST is done with the likelihood ratio test statistic for the fit of each specification of the model to the data. The ratio ) ) 8) (2-29) 1 = L‘ L( Q) is formed where L(&) is the maximum value of the log like- lihood (2-28) in the restricted parameter space specified by the null hypothesis and L(§) is the maximum value of the log likelihood in the unrestricted parameter space specified by the alternative hypothesis. The test statistic equal to -2 1n (1) is distributed according to the central chi square distribution with degrees of freedom equal to the difference of the dimensionalities of the parameter Space 0 and the number of parameters estimated under the null hypothesis. If the test statistic is greater than the tabled chi square value, the null hypothesis of fit of the model is rejected. In the situation in which estimation of the param- eters for Z and for pé is necessary, hypothesis testing is completed in two stages. First, estimation proceeds for £5 33 T, and W given certain restrictions on T and W. The likelihood ratio test statistic, T is then found for 1- the resulting estimates. If the null hypothesis is not rejected, the restrictions placed upon T and W are rea- sonable and one assumes the model fit. The procedure is repeated again, restricting those elements of E6 which represent the repeated measures effects, to be zero. Again the likelihood ratio test statistic, T2, is calculated. If the additional restriction on 26 is reasonable, e.g., the null hypothesis of no repeated measures effects is true, the second test statistic will be similar in value to the first. If the null hypothesis is not true, the lack of fit of the restricted p§ will show in the larger chi square test statistic. Thus, the difference between the two test statistics, T1-T2, is of interest as it forms a new test statistic for the test of the null hypothesis concerning £5. The difference between two chi square values is also dis- tributed as a chi square distribution with degrees of free- dom equal to the difference in df for T1 and T2. Rejection of HO implies that there are repeated measures effects. CHAPTER III REVIEW OF THE LITERATURE Whether or not one's data exactly meet the assumptions of a model is not necessarily the deciding factor in choosing a model. A relevant question to be asked is whether plausible violations of the assumptions affect the validity of probability statements made with respect to the status of the null hypothesis. Some statis- tical tests are robust or insensitive with regards to one or more of the assumptions underlying the test. Simulation is a procedure used to investigate such robustness properties. The distributions of the test sta- tistic are of interest. One needs to know if deviation from an assumption will change this distribution which, in turn, will affect the probability levels normally associated with the test statistic. Computers make it possible to generate a large number of samples of size n from a specified dis- tribution, compute the test statistics, and record their distribution. If an assumption of the model is violated, e.g., the parent population is exponential when the model specifies normal, and the distribution of the test statistic 34 35 does not change, the statistic is said to be robust with respect to the violation of that assumption. Many such studies have been completed with different statistical models. The information is useful for a social scientist designing research. If a test has been shown to be robust, a researcher would still have confidence in his results even though he knew the data did not completely meet the requirements of the assumptions. Results of simulation studies are also used to investigate power associated with a given test statistic. If a researcher has to make a choice between two test statistics which are appropriate to the hypothesis of interest, and have been shown to be equally robust with respect to violation of assumptions involved, additional information of the power for each statistic is beneficial in making the final decision. Simulation is not the only method which has been used to investigate properties of statistical models. Exam- ples of data collected as part of research projects are analyzed by different statistical models appropriate to the design. The results of each analysis, such as estimates of effects, length of confidence intervals for the effects and decisions made with respect to the null hypothesis, are compared. Studies such as these demonstrate research situations in which the models are used and how the results are interpreted. This can be useful information for other 36 experimenters with similar problems. The conclusions from such a study are limited to the situation at hand whereas results of a simulation study generalize to a broader range of research problems. The proposed research employs simulation as a pro- cedure for investigating properties of statistical models used in the analysis of repeated measures data. Studies utilizing these techniques for the repeated measures model will now be reviewed to give background information on what has been done in this area and to show how the present study relates to this previous work. SimulationiStudies of Repeated Measures Designs Collier, Baker, Mandeville, and Hayes (1967) come pleted a simulation study of Type I error probabilities for three procedures used in repeated measures designs. The test procedures are: 1. ANOVA of mixed model--unadjusted 2. ANOVA reducing df by a factor e--adjusted 3. ANOVA reducing df by a lower bound of e--conserva- tive. s is a correction factor proposed by Box (1954) and Greenhouse and Geisser (1959). The assumptions concerning the covariance matrix of the latent random variables were the main focus of this study. Random vectors which were multivariate normally 37 distributed with specified mean and covariance matrix were generated. The covariance matrix was specified to vary from homogeneous to heterogeneous with five different patterns of correlation coefficients. When the means of the populations are specified to be equal, rejection of the null hypothesis corresponds to the occurrence of a Type I error. For each of the samples generated, the F ratio was calculated and a frequency distribution of the F ratios formed. Cutoff points for rejecting the null hypothesis were computed according to the three procedures for a specified a level. The number of values beyond the critical value was then compared to the number which would be expected by chance. Agreement between these values when the structure of the covariance matrix deviated from that assumed would suggest robustness of the test for violation of this particular assumption. Results of the simulation showed that with rela- tively homogeneous covariance, the results of the last two procedures are consistent with those of the first. When the variances are unequal, the unadjusted test overestimates the probability of Type I error. The second corrects for this bias whereas the third test is on the conservative side except where matrices are extremely heterogeneous. Davidson (1972) compared univariate mixed model ANOVA to MANOVA of RM with respect to power with a one way design and p measures for n subjects. In this case, data 38 were generated from a population with specified mean vector and covariance matrix. The structure of the covariance matrix was a major concern and two cases were chosen--a covariance matrix which met the assumptions of the mixed model and one that did not. The mean values chosen for the cells in the design were not equal. The degree of inequal- ity between means was specified by the noncentrality param- eter T given by ‘p = TL.“ 1 (“37’“)2' o (1-0) 3—1 where 02 and p are the population variance and correlation of the p measures, uj is the population mean for the jth experimental condition, and u is the overall mean. Now failure to reject the null hypothesis corresponds to the occurrence of a Type II error. Power is the complement of the probability of a Type II error, 8. For each sample the F ratio was calculated and a decision made with respect to the null hypothesis. From the distribution of the F ratios, power was calculated. If the covariance matrix meets the assumptions of the model and the null hypothesis of no difference among the measures for each subject is false, the F ratio for the uni- variate and multivariate procedures both have noncentral F distributions and a common noncentrality parameter. Power was found for three different values of T and four different 39 sample sizes. When n > p-+20, the multivariate test was as powerful as the univariate one. For fewer n, the univariate test has a higher power. Modification of the univariate procedure by the Greenhouse and Geisser adjustment factor was the most conservative test and also less powerful of the procedures. When the assumptions of the mixed model do not hold, power is dependent upon the pattern of the correlation matrix for the p measures. For example, if the experimental conditions divide into two sets where there is high correla- tion of variables within a set and low correlation between sets, the univariate test is always somewhat more powerful than the multivariate test. At the other extreme, when all means differ only slightly but there is a reliable differ- ence between them, only the multivariate test was able to detect the differences. In the intermediate cases, the multivariate test was also more sensitive to differences. The recommendation of the author was to use the multivariate technique if n is large and the assumption of homogeneity is not met. Otherwise use the mixed model ANOVA. Robustness of ANCOVST with respect to deviations of the assumption of multivariate normality was investigated by Bramble and Brown (1972). The data generated were of two types. One was multivariate normal with a variable percent- age of outliers. An outlier is defined as an observation two standard deviations above the mean. The second type of 40 data conformed to the multivariate gamma distribution. Another concern was the structure of the sample covariance matrix so the data also conformed to the following three cases: 1.. Latent variables uncorrelated, error variance homogeneous 2. Latent variables uncorrelated, error variance heterogeneous 3. Latent variables correlated, error variance heterogeneous. The conclusions showed that the procedures are robust with respect to quite large percentages of outliers (20%) and with respect to moderate general distribution deviation as defined by the gamma distribution. Another simulation study of the ANCOVST model (Wiley, Schmidt, and Bramble, 1973) looked at the degree of accuracy with which the maximum likelihood estimates are able to reproduce the corresponding population parameters. Data were generated from a multivariate normal distribution in which the parameters of the structural model for 2 (2-25) were knoWn in advance. The sample covariance matrix of the data was calculated and this matrix then used as input to the ANCOVST routine to estimate parameters and test the fit of the model. In all three cases of this report, the model specified did not show lack of fit and the parameter esti- mates were close to those chosen a priori. 41 Since multivariate analysis of variance has been suggested, although it is not truly appropriate for repeated measures data, a study by Hummel and Sligo (1971) is of interest. Multivariate normal data were generated and then analyzed by three approaches. The situation is that as described by a one-way analysis of variance where each subject is measured on two or more dependent variables and the experimenter is interested in mean level differences across the k cells on these p variables. The three methods of analysis were: 1. A univariate analysis of variance on each of the p variables 2. An overall, or multivariate, test (Hotelling' s T2 ) on all variables simultaneously followed by a univariate analysis of variance on each vari- able separately only when the null hypothesis is rejected. 3. An overall test on all variables simultaneously, as in approach 2, followed by a simultaneous confidence interval procedure when the null hypothesis is rejected. The variables of interest were error rates per comparison, experimentwise error rates, and the average number of errors in experiments with at least one error. The error rates were defined as the number of times one rejects the null hyp6thesis of no difference of the means. The recommendations of the authors discouraged univariate ANOVA with multivariate data. The unknown experimentwise error rate allows for misinterpretation of the data. The third method of multivariate overall test and 42 simultaneous confidence intervals is extremely conservative and is useful only when the cost of a Type I error is extremely high. The second method, called the combination approach, is considered the best as it gives an experiment- wise error rate which is consistent but yet this test is not as conservative as the third method. Data Examples of Repeated Measures Designs Danford, Hughes, McNee (1960) had a set of data in which two groups of subjects were measured on one dependent variable over ten points in time. They were interested in differences over the time points which suggests using a repeated measures model for the analysis. The authors chose two such models--univariate mixed model ANOVA and MANOVA. The first step was to test their data for equal variances using Wilks likelihood ratio. The hypothesis of equality of variances was rejected (p‘<.001). Thus the assumption of the traditional mixed model ANOVA did not hold. The data were then analyzed by the two procedures. The same conclu- sions were drawn from the mixed model ANOVA and MANOVA. A data example by Elston and Grizzle (1962) employed a one sample design where each subject had been measured over four points in time. Analysis was performed by gen- erating confidence intervals for differences between measures using the following three methods: 43 l. Multivariate approach 2. Complete independence approach 3. Mixed model approach. The multivariate approach assumes independence between subjects but allows for observations on the same individual to be correlated. The second approach assumes independence between subjects and independence of measures on the same individual. Method 3 assumes that the observations for a subject are equally correlated. With all three models, the estimates of parameters of interest are similar. The differences between them are demonstrated in the test of the model and the width of the confidence intervals around the estimates. Method 2 gives the narrowest band but should not be used unless one meets the assumptions of the model. The bands of Methods 1 and 3 are similar which gives support that the assumptions of equal correlation are met. The recommendation of the authors is to use Methods 1 and 3. Cole and Grizzle (1966) compared univariate pro- cedures with that of MANOVA of RM. There were four groups of subjects in the data example, i.e., one way classifica- tion with four levels, and four measures of the dependent variable. The mixed model ANOVA was performed but labelled inappropriate by the authors after the results of the like- lihood ratio tests. The hypothesis that the assumptions of mixed model ANOVA were met was rejected (p‘<.001). The data 44 were then analyzed by MANOVA of RM. The same conclusions were reached by both procedures. Data examples for ANCOVST have also been reported. One study (Wiley, Schmidt, and Bramble, 1973) evaluated the sources of variance in scores on a test designed to assess teachers' judgments in classroom situations. The design over the repeated measures was a 23 factorial. The sample covariance matrix was found. First, the restricted model- was fitted to the data. In this case, the error variances are assumed homogeneous and the latent variables uncorre- lated as in the classical mixed model. The parameters were estimated. The likelihood ratio test statistic indicated a significant lack of fit for this model. The next model specified the latent variables to be uncorrelated and the error variances to be heterogeneous. Again the model did not fit. The third model assumed the latent variables were correlated and the error variances homogeneous and showed significant lack of fit. When the latent variables were specified as uncorrelated and the error variances hetero- geneous, the fourth model, the likelihood ratio test sta- tistic indicated that the model fit. The values for T and W estimated in this case were then interpreted in order to locate the sources of variance in the given problem. Schmidt and Scheifley (1973) estimated parameters for the same four models as above for a set of data with a 3 x 3 factorial design over the repeated measures. A lack 45 of fit was found in the first three models indicating that restrictions on the structure of the covariance matrix are not always realistic. The fourth model did not indicate such lack of fit. The Present Research The study of this paper utilizes simulated data to investigate assumptions of the repeated measures model. It is an extension of previous work and the first to compare the three procedures described in Chapter II. The results will present a more complete picture of Options available to the-experimenter when a repeated measures model is appropriate. CHAPTER IV SIMULATION PROCEDURES Three statistical models have been proposed for the analysis of repeated measures data. Each model differs in its assumption concerning the structure of the covariance matrix of the latent variables. Mixed model ANOVA is the most restrictive and requires independence of the latent random components of the model, i.e., T being constrained to be a diagonal matrix. MANOVA of RM is the least re- strictive and assumes T is a positive definite matrix of the same rank as 2 which implies correlated latent random components equal in number to the manifest variables. ANCOVST is flexible in that one can specify a T to conform to the constraints of the mixed model or to be as general as MANOVA of RM but one which has fewer latent components thus allowing for a more parsimonious model. (This is in the 22 example equivalent to fitting only a main class model.) The general question of the present study concerns the properties of the respective statistics of the three models for repeated measures analysis when data are gen- erated from a population in which the parameter matrix T is known to (1) meet the assumptions of that model and (2) 46 47 violate the assumptions of that model. Simulation is the method used to explore this question. This gives the advantage of knowing population parameters and one can compare sample estimates to the parameter values. Studies of a similar nature to the present one were reviewed in Chapter III. Up to this point in time, most of the work has concentrated on the mixed model ANOVA. A few simulation studies (Davidson, 1972) have compared mixed model ANOVA to MANOVA of RM. Although researchers have investigated properties of ANCOVST (Bramble and Brown, 1972; and Wiley, Schmidt and Bramble, 1973), no one has compared all three models for repeated measures analysis as in the present study. Specific questions of interest concentrate on hypothesis testing and estimation. For hypothesis test- ing interest centers on the effect on Type I and Type II errors of violating the assumptions of a specific procedure. The key issue in the estimation phase concerns the bias of the estimators, especially for ANCOVST. The procedures employed in the study to answer these questions will now be discussed. First, a description of the population parameters E and Z and the manner in which they were chosen will be given. The second section will describe the computer routine utilized to generate the data followed by a description of tests performed on the genera- tion routine. The fourth section looks at the computer 48 routines to analyze the data according to each model. Finally, the measures of Type I errors, Type II errors, biasedness, and efficiency will be described. Description of Population Parameters The data generated for the present study are from a multivariate normal distribution with a mean vector E and a covariance matrix 2. There is one group of subjects, i.e., a single sample, with a 22 factorial arrangement over the four repeated measures (see Figure 2-2). For each subject (N==30) it is necessary to generate four observations, one for each cell in the design over occasions. Three population covariance matrices were chosen such that each was appropriate to the assumptions of one of the three analysis routines. Since each model specifies a different structure for T, 2 was calculated according to the equation 2 = KTK5+W where T was chosen to represent each analysis situation. For the mixed model ANOVA, the latent variables are assumed to be orthogonal so that Z is con- structed with T as a diagonal matrix where the diagonal elements represent the variances of the latent variables.. For the covariance matrix appropriate to MANOVA of RM, T is a general positive definite matrix of the same rank as 2 which means all latent variables are included. ANCOVST allows the experimenter to fit only the main class model to the data so T is specified as a 3 x 3 matrix, allowing for 49 correlated effects but only including the three latent variables appropriate to the main class model as shown in Table 2-5. In each case, T was considered to be homogeneous. Three population mean vectors were also chosen. The first was designed with all means equal so that the null hypothesis of no repeated measures effects was true. However, repeated measures effects were made to be present in the other two choices of g. In one, there was only an effect due to the first factor in the occasions design. The other choice for E had effects due to both of the repeated measures factors as well as to their interaction. The size of the effects was selected so as to give theo- retical power values within the range of .3 to .97. The theoretical value of power was found in tables given in Morrison (1967) for a specified noncentrality parameter, ¢ where 6 d): MI and 52 = g' zgl g. The three choices of Z and p_form a 32 crossed design of possible combinations of population parameters (Table 4-1). One thousand samples of thirty subjects each were generated for each cell in Table 4-1. 50 UIHHH mIHHH m-HHH UIHH mIHH m-HH 0-H m-H «1H uoommo as .m .8 haco uoommo o uoommm coflmoooo oz 0 m d mcofiuoasmom mo daemon um m use a mo onsuosupm Homoeow Hlv mamma Hmuocmmulzm mo ¢>oz¢z How museumoummm w Hooos mmoao owns Bm>ooz< Mom museumoummm N ¢>oz< Hence coxwa Mom oumemoummo w HHH HH 51 An empirically generated example served as the basis for the numerical selection of T and T. Repeated measures data collected by Miller and Lutz (1966) with a 23 factorial design were analyzed by ANCOVST with a model fitting the main effects and the two way interaction terms. From the resulting parameter estimate of T it was possible to form a number of choices of T for a 22 design by choosing the grand mean effect, 2 of the 3 main effects and l of 3 interaction terms. The value for T was selected in a similar fashion. The final selection of T and V and the resulting Z is given in Table 4-2. The mean values of the data from the Miller-Lutz example centered around 20 so that value was chosen for the grand mean of the generated data. Effects were con- structed based on a mean of 20 to achieve the desired level of power. The three mean vectors and their respective theoretical powers, where applicable, are given in Table 4—3. Description of the Generation Routine The present situation requires data to be generated from a population in which the repeated measures are dis- tributed normally with a known mean, E! and a known covar- iance matrix,£,as described in the previous section. Three main steps comprise this generation procedure: 1. Generation of independent random variables which are uniformly distributed between zero and one. 52 mo.m o o o mo.m o e u s mo.m e Row-3033 mo.m F l mommo .38 .5 .I .l l. J eom.~m Nee.” eme.ea mem.a omm. mea.- Mme. mac. . eee.ea ooe.m eme.ma eam.m omo.H mem.- am no asozaz HHH emm.ma em~.m . woe.e Ame. mueeumoumme w xeeuumassee oao.m~ eo~.m rl I1 [I L fl ema.- mmm.m eee.ea emm. J .1 J fleece eam.ea ema.m Nem.ma eam m mwm.w www.- eeufle cues HH one.ma ~ma.m eo~.m -am>ooz< 0» Aofluuofishmv mam.~m museumoummm w I l r I -e.o~ oeo.e www.mH emm.m . om~.o o. e e . . . . a>oz¢ Hoeos .. en. New .... w H o mammoummo Reassesssee mae.om eo~.m u . m —' I. r L IN- mv. omuo move-gum: oosowuuaroo cofiumasmom va Wanda 53 TABLE 4-3 Population Mean Vectors and Power Power Case _6 E Z . 05 . 01 "20.0 7 "20.0 7) (not applicable) A 0 20.0 0 20.0 0 20.0 L .— ... ..- 520.0 7 720.757 I .67 .40 B .75 20.75 II .73 .49 0 19.25 III .73 .49 0 19.25 L _ _ _ ”20.0 T ”20.0'1 I .68 .41 .75 21.5 II .78 .54 C -.25 19.5 III .78 .54 -.50 19.0 )— - — _ 2. Combine the uniform variates so as to obtain normal deviates with a zero mean vector and an identity matrix for a coVariance matrix. 3. Transform the normal deviates to obtain the desired structure of p and 2. Each of the three steps is now discussed in detail. A mixed congruential generator was used to obtain the uniform random variates as described by Gordon (1969) and adapted to the CDC 6500 by Sidney Sytsma. Starting with an initial number, called the seed, a second number is generated which is then input to the production of a third number, and so on. Given three constants A, u and 54 P, the procedure derives the (i+l)th number from the ith number by multiplying by A, adding u. and then taking the remainder, or residual, upon dividing by P. (4-1) G = (Aci+u) mod (P) i+l The seed or the initial number is co. The values of P, 1, u, and co which are chosen so as to maximize the period of the generator, produce numbers that behave as if they are random, and are independent of each other. The second step in which the values from the uni- form distribution are converted to values from the normal“ distribution employs Teichroew's method to approximate the inverse of the probability function for the standard normal distribution. The inverse procedure is a commonly used way of deriving continuous, non-uniform number sequences from uniformly distributed sequences. If f(x) is a probability density function, and its cumulative distribution function, F(xL is non-decreasing and lies between 0 and 1, each number of the sequence from a uniform distribution is considered to be a value of the function F(x) and the corresponding value of xi is determined. In other words, the inverse of the cumulative distribution function is evaluated with a sequence of uniformly distributed random numbers. The cumulative normal distribution, F(x), x 2/2 (4-2) F(x) = -l-I e-t /2_1r -oo dt 55 or its inverse cannot be expressed in terms of simple mathematical functions, so approximation methods are necessary. Teichrow used a polynomial approximation to eval- uate the inverse function. His procedure generates 12 independent random variables, U1, U2, ..., U32, uniformly distributed between zero and one. Then, R is defined such that (4-3) R= (U1+U2+...+U12-6)/4. The normal deviate, z, is then approximated by (4-4) 2 = ((((a,R2 + a7)R2 + a5)R2 + a3)R2 + a1)R where a1 = 3.949846138 a3 = .252408784 as = .076542912 a, = .008355968 a = .029899776 (Knuth, 1968). Extremely large values of z are never obtained but the probability of getting values larger than this method rightfully gives is less than SUEUU" Since each observation needed in this study consists of four measures, the procedure is repeated to obtain a 4 x 1 vector z_which is normally distributed with a mean vector of zero and an identity matrix as the covariance matrix. 56 The third step transforms 3,to y_where y_is distributed (3,2). The transformation necessary is (4-5) y_= Tz + p where T is the cholesky factor of Z. The cholesky factor is a lower triangular matrix such that TT' = Z. This is used because the covariance matrix of the transformed variates (4-6) y’= TE + E is (4-7) Var(y) = T var(g)T'. In this case, Var(£) is the identity matrix so (4-8) Var(y) = TT' = 2 which gives the desired result (Morrison, p. 79). The addition of E_in (4-5) only changes the point of central tendency for the distribution of y, After the transforma- tion, ylis distributed normally (p,2). The generation program, GENDATA (see Appendix), completes each of the three steps as one observation is formed. The sample size chosen is 30 so 30 such vectors y comprise one sample. The three analysis routines do not require raw data but begin with the summary statistics of the sample mean and sample covariance matrix. Thus, for 57 each sample, the mean vector, 2, and the sample covariance matrix, S, are computed to be used as input in the analysis phase. For each case described in the previous section, 1000 samples are generated. Test of the Fit of the Generators One of the main purposes of simulating data is to obtain sequences of numbers which behave as if they are random. It is also important that their sequences of random digits meet the criteria specified, such as being representa- tive of the uniform distribution. There are many tests that can be used to determine if this is the case. A sample of such tests was applied to the generator used in the present study. The tests of the generator for uniformly distributed number sequences were carried out by Sidney Sytsma. The chi square or frequency test is a basic method to test the resulting distribution of the generated numbers. A sample of 1000 values was simulated. The interval from 0 to l was divided into five parts--each of length .2. The number of values falling into each of the intervals was counted. Given that the values from the uniform distribution all have equal probabilities of occurring, one would expect equal numbers of values, 200, in each interval. The chi square test of goodness of fit was used 2 (0-12)2 (4-9) x2 = E 58 where O is the observed frequency of numbers and E is the expected frequency of numbers. Degrees of freedom equal the number of intervals minus one or 5-1==4. Results of the frequency test applied to the 1000 generated values showed a chi square of 4.4 which with four degrees of free- dom is not significant at the .05 level and the hypothesis of uniformity is not rejected. To insure the pairs of successive numbers were uniformly distributed in an independent manner, a serial test was applied to the same sample generated above. The 1000 values generated were divided into 20 sets where there were 50 values in each set. A series of chi square tests were made based on the frequencies of patterns reoccurring in each of the 20 sets of numbers. The results were then combined into an overall test of serial randomness for all values. The hypothesis of randomness was not rejected at the .05 level. After the second stage of the simulation process, a chi square test was again run by the present author on the normal variates in order to see if Teichrow's approx- imation method had indeed given sequences which were now normally distributed with a mean of zero and a covariance matrix given by the identity matrix. The abscissa of the theoretical normal curve was divided into 82 intervals. Each interval between -4 and +4 standard deviations was of length.1. The other two intervals covered values less than 59 -4 and greater than +4 standard deviations from the mean. Based on the area under the curve for each interval, expected frequencies were computed for a sample of n==5000. A sample of this size was then generated and the number of observations actually in each interval counted. A chi square test statistic was found using (4-9). Its value, 98.497, was not significant at the .05 level so the hypoth- esis of fit was not rejected. It is also possible to test whether or not the sample covariance matrix, S, calculated after the three steps in GENDATA is from a population with covariance matrix 20. The null hypothesis is tested against the alternative H1: 2 # 20. The test statistic is N/2 l (eokN/Z e-1/2 tr (82' ) -1 N | (4—10) A = |Bz where B is the sum of squares and cross products matrix, k is the number of repeated measures, and N is the sample size. The value -ZlogA is distributed as a chi square with k(k+l)/2 degrees of freedom (Anderson, pp. 264-267). Ten sample covar- iance matrices calculated from the transformed observations were submitted to the test. Results are given in Table 4-4. 60 TABLE 4-4 Test of Fit for Sample Covariance Matrix for N=30 and K=4 2 Sample x Df p 1 5.7723 10 .90 2 11.4029 10 .50 3 11.0772 10 .50 4 12.6107 10 .25 5 6.7376 10 .75 6 17.1957 10 .10 7 9.8471 10 .50 8 25.3335 10 .005* 9 8.2654 10 .50 10 16.5647 10 .10 *Significant lack of fit. Although there are other measures available to judge the effectiveness of a generation process, it was felt that the results of the tests made on the generator have shown it to be at least satisfactory. The process appears to be random and to have the distributional characteristics that are intended. Analysis Routines Output from the generation program consists of a vector of sample means and the corresponding sample co- variance matrix for each sample. This serves as input to the two analysis routines. 61 The first routine, ANALSIS, performs two analyses: MANOVA of RM and the mixed model ANOVA. The matrices needed, SSM* and SSE* which are described in Chapter II, are computed. In order to test the null hypothesis of no differences across the repeated measures, the determinantal equation of (2-17) is solved and the A of (2-18) is computed from which the multivariate F ratio due to Rao (1965) and its probability level are calculated. The routines used for this latter stage of the analysis program were taken from a multivariate analysis of variance program, MULTIVARIANCE, written by Jeremy Finn and adapted for use on the CDC 6500 at Michigan State University by Scheifley and Schmidt (1973). The same hypothesis is then tested using the classical ANOVA model where the necessary sums of squares are found by combining the appropriate elements of SSM* and SSE*. The second routine, ANCOVST, performs the analysis of covariance structures. The parameters of the model, u , T, and W, are estimated by the method of maximum likelihood. The set of routines used to estimate these parameters by an iterative procedure were developed by Fletcher and Powell (1963). The fit of the model estimated is then tested via a likelihood ratio test statistic. The covariance matrix of the estimates is also available. Many of the matrix operations necessary for ANCOVST were done by subroutines 62 written by R. Darrell Bock (1967) and adapted for use on the CDC 6500 by Schmidt (1970). Measures of Type I and T e II Error, Biasedness, and Eff1c1ency An experimenter wants to choose an analysis pro- cedure which is powerful enough to detect differences in mean levels if they exist and in which the level of sig- nificance is accurate, not inflated or conservative. One empirical question is what effect does the structure of 2 have on Type I and Type II errors when the data are analyzed according to each of the three models used for repeated measures data? When data are generated from a population in which all means are equal and the null hypothesis of no repeated measures effects is rejected, a Type I error is made. An empirical estimate of the probability of a Type I error is computed by tabulating the frequency with which the null hypothesis is rejected at a specified a level by each model for those samples in which the population means are equal (cases I-A, II-A, III-A in Table 4-1) and dividing by 1000, the number of samples in a group. Two specified 0 levels are selected for the present study--that of .05 and .01. In two of the models, MANOVA of RM and ANCOVST, where there is one overall repeated measures hypothesis test, the number of rejections with a‘<.05 and a<<.01 is recorded. In the mixed model ANOVA, there are three separate tests--one for 63 factor A, one for factor B, and one for the interaction of factors A and B. For this model, counts are made of the number of times the null hypothesis is rejected in the following manner: 1. for each of the three separate tests at the .05 and .01 levels; 2. for any of the three separate tests at the .05 and .01 levels, i.e., a rejection is noted if one or more of the three tests have a probability level less than the specified levels; 3. for each of the three separate tests at the .0169 and .0035 level which gives an overall a level of .05 and .01, respectively;1 4. for any of the three separate tests at the .0169 and .0035 levels. When the pOpulation means are not equal and the null hypothesis is not rejected, a Type II error is made. An empirical estimate of the probability of a Type II error is computed by tabulating the frequency with which the null hypothesis is not rejected at a specified a level by each model for those samples in which the population means are not equal (cases I-B, II-B, III-B, I-C, II-C, III-C in Table 4-1) and dividing by 1000. Power is equal to l-P(Type II error). This Count is made at the"levels described for 1These values were calculated from 0* = l - (l-ai) 1 lit—duo i where 0* = .05 and 8* = .01 are of interest. 64 each of the Type I errors. Thus, estimates of Type I and Type II error are found for each analysis procedure after completing the appropriate hypothesis test described in Chapter II. Each of the three analysis routines has its separate criterion for hypothesis testing. In order to compare all three models on a similar scale, the asymptotic chi square test was also used in the study to test the null hypothesis of equal means among the repeated measures. This procedure is based on the least squares estimates of the repeated measures effects and their estimated variances. For MANOVA of RM and mixed model ANOVA, calculation of this test sta- tistic is similar. The least squares estimate of effects, 8, for each sample is given by (2-16). The variance of‘é is A _ A _ —l/\ —l ' (4-11) Var(§) — Z — K 2 (K ) . ICD) M) In the MANOVA of RM model, is estimated by the sample covariance matrix. In the mixed model ANOVA, E is a diag- onal matrix with diagonal elements equal to the variances of each repeated measure. The least squares estimates of effects for ANCOVST, pg, are given in (2-24). The variance of ya is l 1 (4-12) Var(p_‘é') = 33 = (K'K)— K' §K(K'K)' E5 65 where E is found from the estimated values of T and T for a given sample according to (2-25). Since the interest here is only in the repeated measures effects, the next step in the asymptotic chi square test is to partition the vector of effects and its covar- iance matrix so as to omit the grand mean term. For the given example, 2c is a 3 x 1 vector and Hg has two terms. c The covariance matrices, 28 and Zfi~ , are now 3 x 3 and c e . 2 x 2, respectively. The test statistic, C, for MANOVA of RM and mixed model ANOVA is _ .. -1 A (4-13) c — _c 22c 51c. For ANCOVST, C is (4-14) c = is 2 y where, in both cases, C is distributed as chi square with degrees of freedom equal to the number of parameters in the vector of effects. Estimates of Type I and Type II error are computed for the asymptotic chi square test in the same manner as has already been described. The issues of biasedness and efficiency concentrate on the estimates of the parameters of each of the models and their standard errors. An estimator is unbiased if its expected value is equal to the population value of the 66 parameter. The estimator with the smallest standard error term among the set of unbiased estimators is said to be relatively more efficient. The values of the parameters for each of the models are known (Tables 4-2 and 4-3). Estimates for MANOVA of RM and mixed model ANOVA, §,'are found by the least squares estimation technique and their value is the same for both models. The standard errors of these estimates are equal to the square roots of the diagonal elements in the matrix 2 in (4-11). Maximum 0 likelihood estimation procedures are utilized in ANCOVST to estimate pé, T, W. This iterative estimation procedure generates an estimate of the covariance matrix of the estimates. The square roots of the diagonal elements of this matrix are the standard error terms of the estimates. As each of the nine groups of data are analyzed by all three models, the mean values of the parameter estimates over the 1000 samples are computed. If the estimator is unbiased, the mean value should closely approximate the known population parameter. Thus, by comparing the empir- ical mean estimate of the parameter found in each model to that of the known population value, one can determine the degree of biasedness, if any, present in the estimation procedure. For each of the two estimation procedures, least squares and maximum likelihood, three estimates of the 67 standard errors are also computed. The first is found by averaging the standard errors calculated as part of the analysis procedure over the 1000 samples. The computational formula for the variance of an estimator of §_is used for the actual empirical estimate of the variance. The formula for the unbiased variance is 2(Si-§)2 nz6i-(zéi)2 (4’15) Var(8i) = -—1;:f-—'= n(n:1) and is computed for each group of 1000 samples. The stand- ard errors are equal to the square root of each variance term. The third measure of standard error is similar to the second only it takes advantage of the fact that the population value is known. This formula is A _ 2 Mei e) n (4-16) Var(§i) = and is also computed across the 1000 samples in each of the nine populations described in Table 4-1. The resulting standard errors from the three measures for the two esti- mation procedures are compared. Since the pOpulation value of the standard error for each parameter is known, the three measures above can be compared to the actual value. CHAPTER V SIMULATION RESULTS The simulation procedures employed in this study to investigate the three statistical models appropriate to repeated measures data were reviewed in Chapter IV. The main purpose was to investigate the effect that the struc- ,ture of the covariance matrix of the latent variables had on the results for hypothesis testing and estimation for each of the three analysis procedures. The specification of three structures for Z and three mean vectors E gives a total of nine populations as a basis from which the ques- tions of interest were explored. For each population, 1,000 samples of size 30 were generated. Each group of data was then analyzed by the mixed model ANOVA, MANOVA of RM, and ANCOVST. The results of these analyses are now discussed. First, the probability of a Type I error and the power of each statistical test will be presented followed by the estimates of the parameters and their standard errors. 68 69 Results for the Hypothesis Tests Empirical estimates of the probability of a Type I error and the power of the statistical test for each analysis routine across the nine populations are reported in Table 5-1. These estimates are based on the overall hypothesis tests for MANOVA of RM and ANCOVST each of which was discussed in Chapter II. For mixed model ANOVA, the 8 levels and power are those which most closely correspond to an overall test for the model--from the frequency count for any of the three separate tests at the .0169 and .0035 probability levels giving overall levels of .05 and .01, respectively, assuming the tests to be independent. The standard errors for the empirical estimates of a and the power are estimated by l§q7fi'where p is the theoretical probability of a Type I or Type II error, q equals l-p, and n is the number of samples. The standard errors are reported in Table 5-1. When the assumptions of the mixed model ANOVA are met and there are no repeated measures effects (case I-A), MANOVA of RM and ANCOVST are more likely to reject the null hypothesis than the mixed model ANOVA at the .05 level. The rate of rejection is similar at the .01 level for all three models. When 2 is not consistent with the mixed model assumptions (cases II-A and III-A) there are only slight differences in the empirical 0 levels of MANOVA of RM and 70 .2m «0 ¢>oz¢z Mom mcoeumaammm on» on museumoummmnuHHH uem>ooza How mcowumssmmm Tau on opmaumoummm-IHH m<>ozm Hooos taxes Mom mcoaumasmmm on» on museumoummm-IH “msoaaom mm oocamoo me u xanuma mocoaum>oo ones @530. n on 320. 1 mm .393. u on vovao. u on mamoo. u on mmooo. u on mom. mom. mmm. obs. ooo. Hwo. am>ouz< mob. Hmm. mac. omo. moo. mvo. mm m0 c>oz¢z HHH mum. «mm. Hoe. who. woo. woo. ¢>ozm H0802 ooxaz 2.30. .1. mm oamao. n on .830. u on vovao. u on mamoo. u on mmooo. u on omo. New. oum. one. ado. ooo. Bm>ooz¢ one. cow. vav. ooo. woo. woo. am no ¢>oz¢z HH How. Hmm. mmv. moo. moo. Nmo. m>oz< Hmooz pose: mmmao. 1 mm mhvao. u on mvmao. u on mmeao. 1- om mamoo. u on mmooo. .1- mm «am. new. oov. oeh. Ado. moo. am>oozm moo. new. mom. vow. «Ho. moo. am no «>028: H oom. cam. vow. moo. ado. mmo. m>oz< H0802 ooxaz 3.18 mo.18 3.18 3.18 3.18 mo.18 .3802 MN muoommo Had uoommo 8 memos Anson U ommo m ommu a mono Hmsom Auouno H unhevm memos uonum vumocmum Hausa 8cm mocausom mammamcm Tonga map How Hosom one “woman H ommavm mo moumsaumm Him mflflfla 71 mixed model ANOVA. The empirical estimates of a are smaller than the theoretical value (with the exception of case II—A) implying both tests are conservative when 2 is considered to be a general matrix. For case III-A, conclusions for ANCOVST are similar to those above--it is a conservative test. When 2 is appropriate to the assumptions of ANCOVST as in case II-A, the test of this model rejects the null hypothesis more often than is expected by chance alone. With the exception of the estimate for ANCOVST in case I-A at the .05 level, all empirical estimates for a are within two standard errors of the theoretical values. This sug- gests that the conservative trend of the point estimates may be solely due to chance variation. When the repeated measures effects of the model are slight, i.e. only one effect present, as in cases I-B, II-B, and III-B, ANCOVST is always more powerful in detecting the effect than the other two procedures. Mixed model ANOVA has larger estimates of power than MANOVA of RM at the .01 level. When a==.05 and the mixed model assumptions are violated, MANOVA of RM is slightly more powerful than the ‘ANOVA procedure. When assumptions are met, mixed model ANOVA is the more powerful of the two models. As the effects become larger, i.e. more effects present in the model (case C), the power estimates for ANOVA of RM and mixed model ANOVA increase so as to be close to or larger 72 than those for ANCOVST. MANOVA of RM is now most powerful no matter what the structure of 2. Of the other two procedures ANOVA is more powerful than ANCOVST when its assumptions are met. Otherwise their power estimates are approximately equal. The empirical estimates of power can also be com- pared to the theoretical values stated in Table 4-3. For case B, MANOVA of RM is lower in power than predicted. The same is true for mixed model ANOVA except when the structure of Z is appropriate to the assumptions of the model (case II-A). The power estimates of ANCOVST, the most powerful of the three in this situation, are either larger or very close to the theoretical values for power. When the re- peated measures effects are larger, the power estimates of all three models are greater than the theoretical values. With few exceptions, the estimates of power are outside the bounds of a 95% confidence band for the theoretical values suggesting the trends described are not due to chance. With MANOVA of RM and ANCOVST, one hypothesis test is performed to check for effects over the repeated measures. Mixed model ANOVA has three tests for the same purpose. Thus, the estimated error rates for this procedure are not as straightforward as with the other two. An error rate can be defined for each of the three hypotheses of the model (2-2) or it can be defined for the experiment. The latter 73 case most closely resembles the overall test of the other models. If the error rate is set for each hypothesis and the tests are independent of each other, the probability at least one of the hypotheses will show spurious sig— nificance is equal to l--(l--0L)c where C is the number of hypotheses. This is appropriate when the mixed model assumptions are met because in that case the three tests are independent. Thus when one rejects each hypothesis at the .05 and .01 levels the theoretical probability of at least one error is .1426 and .0297, respectively. Empirical estimates of error rate per hypothesis are recorded in the first two columns for cases I-A, II-A, and III-A in Table 5-2. The other two columns in Table 5-2 represent empirical estimates of a when the probability level for each hypothesis is set lower so as to make the overall level of the experiment equal to .05 and .01, respectively. For case I-A the estimated a levels per hypothesis are larger than the F tables would predict. In case II-A when the assumptions of the model were violated, the empir- ical a level is closer to the theoretical one. As the assumptions were further violated in case III—A, the test per hypothesis for mixed model ANOVA is now conservative. This same trend is present in the a level per experiment-- 0L= .0169 and a: .0035. 74 .zm mo «>Oz u<>oz< HmoOE omxHE Hem mGOHumEsmmm may cu mumHumoummmunH "m30HHow mm omchmo mH w xHHumE mocmHHm>oo mzam Q omnm. nvmm. mob. mvm. QHov. ammo. moo. vmm. Amoo. vio. mmo. vNH. umwu mad Hon. omm. vmm. omm. moo. mmo. vHo. omo. moo. omo. HHo. omo. m x d HHH oHo. mmo. Hmo. moo. moo. oHo. moo. moo. moo. oHo. boo. Hmo. m nouomm now. mum. «mm. mmm. nmv. moo. omm. mHm. voo. oHo. HHo. moo. d Houomm nHoo. nHmm. mph. mvm. ammo. ammo. 5H0. ovo. Qmoo. Ammo. vmo. HmH. umwu and mNo. vow. Hum. hHo. ooo. mmo. mao. Hmo. moo. mHo. «Ho. moo. m x 4 HH «Ho. vmo. mmo. «no. moo. mHo. moo. mvo. moo. Hmo. mHo. mmo. m Houomm how. mmo. 5mm. 5mm. Hmv. mho. moo. mmm. Hoo. oHo. woo. vmo. m Houomm noom. beam. omh. vmm. viv. Ammo. mHo. 5mm. QHHo. Ammo. omo. ooH. umwu >s< omH. ohm. mom. vow. ooo. mHo. oHo. boo. moo. mHo. NHo. vmo. m x m H moo. mmo. mmo. who. moo. vHo. moo. mvo. moo. BHo. oHo. mmo. m Houomm bmv. Hon. HHo. omo. wmo. who. woo. Hmm. moo. mHo. moo. mmo. a Houomm mmoo. moHo. Ho. mo. mmoo. moHo. Ho. mo. mmoo. moHo. Ho. mo. umma mm muommmw HHa uomwmw d msmwfi Hmsmm U wmmu m mmmu m mmmo Hm3om . . “Hound H cameom ¢>oz¢ Hmooz omxflz ou msmwso mmuMEHumm Hm3om can auouum H mmmsom Nlm mamdfi 75 Estimates of power when testing each hypothesis and for the overall tests are also given in Table 5-2. As a levels and power are related, one would expect that the test of each hypothesis at .05 and .01 levels, is more powerful than the overall test at .05 and .01. This is shown to be true by the empirical estimates. Setting an a level per hypothesis may give one a more powerful test but the chances of committing a Type I error across the entire experiment increase. Power of the mixed model ANOVA varies only slightly as the structure of 2 changes and no pattern is discernible. The estimated probabilities of Type I errors and power for the asymptotic chi square test are shown in Table 5-3. This test is included in the present study in order to compare the three procedures on the basis of a common statistical test. Since it is an asymptotic test and the sample size is relatively small, one would not expect the results to be entirely consistent with the normal hypothesis testing procedures for each analysis. The esti- mated a's for each of the three procedures are larger than is to be expected. The exception to this is for the ANCOVST model in case III-A. Thus when using this test one would expect to reject the true null hypothesis more often than one should. .zm mo «>024: How mGOHumssmmm on» on mHMHumoummmnnHHH a9m>oozm How mcoflumesmmm may cu mumHHmoummmulHH u<>oz¢ Hmooe omxHE How msowumESmmm msu ou mumHHmoummmnnH ”m30HHom mm omcfimoo mH w xHHumE mUGMHnm>oo moan 76 mmo.. mam. mHm. «Hm. HHo. amo. am>ooz< who. mmm. mmm. mph. mHo. woo. 2m mo ¢>oz¢z HHH mom. vvm. MHo. ops. Hmo. mmo. ¢>ozm Hmooz owaz Hmo. mom. Hmo. mom. mmo. mno. am>ooz« How. vvm. mum. mun. «mo. mmo. 2m mo ¢>oz¢z HH mmm. hvm. 0mm. huh. vmo. moH. «>oz< Hmooz omtz ohm. mom. mum. own. mHo. who. am>ooz< cub. vom. mom. 5mm. omo. moH. 2m mo «502$: H «mm. mHm. mmm. vow. «mo. NoH. 4>oz¢ Hmooz omtz Ho.18 mo.18 Ho.18 mo.18 Ho.18 mo.18 Hmooz mm muomwmw Had nommmm < momma Hmswm U wmmo m mmmu d wmmu umma mnmcvm H30 caucumawmm mmcflusom mHmmHmaa wanna map How umzom paw Anonum H ommaom Mlm Manda 77 When comparing the three models as to the estimated power for the asymptotic chi square test, the conclusions are similar to those for the other hypothesis testing pro- cedures. When the effects are small, ANCOVST is the most powerful. As the effects increase, MANOVA of RM and mixed model ANOVA increase in power almost identically over ANCOVST. When the power estimates of Table 5-1 were compared to those of Table 5-3, the chi square testing procedure was found to be more powerful than the hypothesis test normally used for each specific model. This, however, could be an artifact of the poorer fit of the x2 test to the null distribution. The hypothesis testing phase of ANCOVST also has a general test of fit for the whole model. The null hypoth- esis specifies a structure for E and Z in terms of the parameters ¢, and W. Maximum likelihood estimates for £5 these parameters can be calculated and the likelihood ratio test performed. If the null hypothesis is rejected, the specifications of the model are not consistent with the data. For example, in case II—A, the null hypothesis states that ¢ is a general covariance matrix for the main class model, W is a diagonal matrix with all diagonal terms equal, and E5 is a function of the main class model only. In this case (II-A) the population parameters were constructed to be consistent with these specifications. If, for a given 78 sample, the null hypothesis is rejected, a Type I error has been made. The empirical estimates of a were found for each of the nine groups of data and are reported in Table 5-4. All values are larger than the theoretical values of .05 and .01. In case II-A and II-B, E5 and 2 were constructed according to the specification of a main class model. That coincides with the specifications made when analyzing the generated data by ANCOVST and so it is expected that the model would in fact fit the data. Although the empirical estimates of a are larger than the theoretical values of .05 and .01 in these two cases, they are smaller than for any other case (with the exception of case II—B at the .05 level). The standard error terms for the estimates of a are .0069 and .0031 for the .05 and .01 levels, respectively. All empirical estimates in Table 5-4 fall outside of the bounds of a 95% confidence band for the theoretical values. Thus one is more likely to reject the fit of the correct model when using the chi square approximation to the likelihood ratio statistic than the theoretical value of a would sug- gest for samples of size 30. In cases I-A, I-B, III-A, and III-B, the interaction term of the factors is zero and the main class vector of repeated measures effects estimated in ANCOVST still applies. But 2 in these populations was con- structed such that the random component of the interaction 79 TABLE 5-4 Test of Fit for ANCOVST Model 4— - a Case A Case B Case C Equal means A effect All effects 2 a=.05 a=.01 a=.05 a=.01 a=.05 a=.01 I .103 .047 .109 .033 .355 .151 II .081 .022 .115 .028 .494 .266 III ...134 .038 .131 .036 .516 .288 term was non-zero. Thus the specification of 4 for the main class model in these cases might create lack of fit. The empirical estimates in Table 5-4 show that the model is rejected only slightly more than in cases II-A and II-B. It would seem that omitting the interaction components of 4 does not seriously affect the a level for rejection of the model. The estimates of a in cases I-C, II-C, and III-C are largely inflated. These situations were designed so that the interaction of factors A and B in the mean vector is non-zero. Since the ANCOVST model assumes this component to be zero, the increase of the lack of fit for this situa— tion correctly implies that this is not a valid assumption. Results for Estimation Phase The repeated measures effects and their standard errors were estimated by two procedures. Mixed model ANOVA 80 and MANOVA of RM used least squares estimation. Maximum likelihood estimates are found in ANCOVST. The results of the two procedures are given in Tables 5-5 and 5-6. For each of the parameters estimated, the following summary statistics for each set of 1,000 samples was computed: 1000’\ 2 a Mean = 9.: 1000 2 se i=1 1 — 1000 ‘iaoofi, _,2 2 (9.-@) i=1 1 2 999 1000 A . 2 (91-9)2 i=1 SE = 1000 average of values for the estimates of population parameters from each sample. average of values for the standard errors of the esti- mates which are calculated as part of the estimation proce- dure from each sample. empirical estimate for the standard error of the esti- mates which is based on the squared difference between the estimates and the mean estimate. empirical estimate for the standard error of the esti- mates which is based on the squared difference between the estimates and the popu- lation parameters. 81 .zm mo «>024: How u$16028 mom x¢>oz¢ Hopes omxHE How mGOHumEdmmm may on muMHMQOHQQMIIHHH mGOHumESmmm on» on mHMHHQOHQQMIIHH mcoHumEdmmm on» on quHumoummman "m3OHH0m mm ongmmo mH w xHHumE moGMHHm>oo maem mmm. mmm. mmm. oom.u own. owm. mmm. moo. mmm. mmm. emm. moo.- coauomumuaH mow. mos. mas. Ha~.- Now. How. mas. mmo.u Nov. mos. wee. mac. «m I in HHH 8mm. 8mm. 0mm. ova. mew. mam. mam. ave. mew. mam. Hmm. moo.- «a u as mam. mom. mom. smm.ma mmm. mmm. Ham. mao.om «mm. Hmm. mom. emo.om cam: gamma mom. mom. mom. mmv.u new. mom. mom. moo. mom. mom. mom. 800.- cowuomumucH mac. «he. see. sem.u Mme. Mme. New. aHo. mas. mew. ens. mac.u m u in HH mam. mam. mam. amp. mmm. mmm. mam. «ma. mew. mew. omm. moo.- «a - Ha Ham. mam. mam. Hmo.om com. com. com. mHo.o~ Ham. mmm. mmm. www.mH :mm: @2618 own. mum. emm. amq.u 8mm. awn. omm. moo. 8mm. 8mm. mmm. moo. :ofluomuwuaH use. mew. ape. mvm.u emv. mma. «be. 5H8. mme. mmv. Nae. 8H0. Mm u in H mam. mam. mam. ems. Hmm. Hmm. 8mm. men. «mm. «mm. 0mm. moo.u a u 8 «mm. omm. 8mm. mmo.om mom. mom. 0mm. mao.o~ mmm. 8mm. 0mm. mHo.o~ cam: osmuo n a H m u H n N H mm mm mm cmmz mm mm mm qmmz mm mm mm :mmz powwow aw muommmw HHfl vacuum 4 mQMmE Hmsmm U ammo m mmmo a mmmo p ¢>oz¢ Hood: omxHZ now am no ¢>ozozooz¢ How £>oz¢ Hmoos omxfla now mGOHuQESmmm on» on muMHHmoummmulHHH mcomuQESmmm map cu mumHuQOHQQMIIHH mcomumEdmmm mop cu mumwumoummmulH umonHom mm omcmmmo mH w xenuma mosmwum>oo moan mme. mme. mme. Hmw.n wme. Hme. ome. mwo.u mme. mme. mme. mHo. Hm I Hm omw. omw. omw. mem. mew. mew. wmw. mem. mew. mew. emw. moo.u 8 n 8 HHH mmm. mmm. mmm. mmm.mw mmm. mmm. mmm. wwo.ow wmm. Hmm. wmm. emo.ow aamz mango a mme. eme. mme. mew.n mme. mme. Hme. mwo. mme. mme. eme. mHo.n Hm I am mew. mew. wmw. mmm. wmw. wmw. mew. wmm. mew. mew. mew. moo.u 8 n 8 HH Hmm. mmm. mmm. Hmo.ow mmm. mmm. mmm. wHo.ow wmm. wmm. mmm. mmm.mH ammz mamum mme. mme. mme. mew.u mme. mme. wme. mHo. mme. mme. Hme. mHo. wm I am mew. mew. wmw. mmm. Hmw. Hmw. Hmw. mem. emw. emw. Hmw. woo.u w8 u a8. H wmm. 8mm. mmm. wmo.ow wmm. wmm. mmm. mwo.ow mmm. mmm. mmm. mHo.ow aamz macaw ”mm wmm amm ammz nmm wmm amm anus mmm wmm aMm ammz pummmm aw muommmm HHm uommmm d momma Hmsmm U mmmu m wmmu 4 ommu Bm>ooz¢ @M.H0m muounm oumocmum cam mwumenumm ooonmamxflq Edfimxmz mlm mflmda 83 The two estimation procedures give practially identical estimates for the repeated measures effects. In the case of the linear model, least squares estimates are unbiased (Winer, pp. 635-638). Maximum likelihood estimates are not necessarily unbiased. Since the estimates are the same in both methods, it can be concluded that ANCOVST gives unbiased estimates of the repeated measures effects given the example of the present study. This con- clusion is upheld when the empirical estimates are compared to the known population values as given in Table 4-3. The standard error terms calculated from the esti- mates of the effects, SE and SE are also identical in 2 3' Table 5-5 and Table 5-6. But the standard errors which represent the average of this value calculated by each routine, SE are different. In all cases, the standard 1: error in ANCOVST is smaller than in MANOVA of RM and mixed model ANOVA for the error of the grand mean and the effect for factor B. For factor A, the standard errors are similar. This implies maximum likelihood estimators are slightly more efficient than those of least squares. As the mean vector and structure of the covariance matrix vary, this pattern remains constant. One can also compare the three estimates' standard errors within each estimation procedure. The values of SE 2 and SE3 are identical. This lends credence to the notion 84 that the estimates are unbiased since the only difference in the formulas between SE and SE3 is the use of the mean 2 of the parameter estimates or the parameter itself. In ANCOVST, SE is usually slightly lower than SE and SE3. 1 2 The standard errors from least squares estimation procedures 2 and SE3. There are exceptions to these conclusions in both Table 5-5 and are usually slightly higher or equal to SE Table 5-6. The population values of the standard errors for least squares estimates were found by taking the square roots of the diagonal elements of Var(§) in (4-11) when 2 = Z. The vectors of population standard errors for E; SE are given below: z .5906 .59067 F.5906 _ .2493 _ .2493 _ .2493 SE21 ‘ .4753 SE21: ‘ .4753 SEZIII ‘ .4753 L.2245 .2051 L°2245 The values of SE1 in Table 5-5 are close to the population values of the standard errors. It is possible to do a similar comparison for maximum likelihood estimation but the procedure involves second derivatives of the function in (2-28) which are not available at this time. The components of 4 and W are estimated only by ANCOVST. It is these estimates which enable the experimenter to investigate sources of variance across repeated measures. 85 1/2 1/2 The cholesky factors of 9 and W, 9 and W , respectively, are found by maximum likelihood procedures. The resulting mean values and standard error terms for each element of 1/2 and W1/2 are given in Table 5-7. The population values 4 for these elements are given in Table 5-8. When comparing the empirical estimates to the population values, one does not find the close agreement as was the case with the repeated measures effects. It seems that the estimates of the variance components tend to be biased. The fact that SE2 does not equal SE3 implies that replacement of the mean value of the estimates by the population value does not give similar results. This supports the notion that these estimates are biased. The estimates of 0 and W are given in Table 5—9. The population values for comparison purposes are in 1/2 1/2 Table 4-2. As with ¢ and W , the estimates do not exactly agree with the known parameter values. Only two standard errors terms are available for this case, SE2 and SE3. 86 .zm mo ¢>oz¢z Mom mcoHumEdmmm ms» 0» mumauQOHQQMIIHHH wam>ooz< How mcofiumfidmmm may ou mummHQOHQQMIIHH a<>ozm H8808 omxms How mcoHumasmmm map 0» muMHHQOHQQMIIH "maoHHom mm omcmeo mm m xwuumfi wocmwum>oo one M wmw. mmw. wmw. mme.w omw. mew. wmw. mwm.w mmw. mew. emw. wem.w was eam.H omw. wmm.e mme. mme.w wwm. mmo.e mmm. mee.w wwm. mmm.m wmm. wMe wmm. wmm. mem. mmm.w Hmm. mmm. Hem. emm.w mmm. mmm. mom. emm.H e mom. mom. eom. www.u mwm. mwm. wom. mmm.- mom. eom. Ham. moa.u ame Haw moe. mwm. mmm. wwm. mmm. mmm. Hem. Hmm. mmm. mew. mmm. mmm. wwe emw. mmw. mmw. eeH. wmw. wmw. mmw. mmw. mmw. mmw. omw. mew. awe mme. mme. wme. mmm.w mme. ome. mee. mwm.w eme. mme. mee. wem.w aae wmw. mew. mew. wmm.w omw. mmw. mew. mmw.w mmw. wmw. wmw. mme.w Ham wme.w mmm. owm.m wem. oew.w mmm. mmo.m mme.w mme.w mmm. ewm.w mme.w mmo mmm. mmm. mom. mmm.w Hem. ewm. mam. mmm.w mmm. mmm. mmm. mmm.w wme mme. mme. Hom. mmm.- mwm. mam. mme. mmm.- wom. wom. wme. mee.- ame HH mmm. omm. mmm. mmm. mmm. wmm. mwm. wmm. emw. wmm. mmm. mmm. «we mmw. mmw. mmw. mmw. emw. emw. wmw. wma. wmw. wmw. mmw. med. awe ome. wee. mee. mmm.w mme. ome. mme. mem.w wme. mee. Hee. mmm.w aae mmm. mmw. omw. mmm.w mwm. mmw. mmw. eoe.w mwm. mmw. mmw. mme.w Ham mme. moe. mmm. wmw.w mee. wmw. emw. wow.w wwe. emw. emm. mmw.w nae H Hmm. mmm. oem. eem. mam. mwe. mmm. ome. mam. awe. mmm. awe. ”we mme. mme. wme. mmm.w mme. mme. mee. wem.w mme. Hme. mee. wem.w e nmm wmm amm ammz mam wmm amm ammz mmm wmm amm ammz umuosmumm aw muomwmo HHfl uommmm a momma Hmsmm U mmmo m mmmo m mmmu N\H? USN H0 HON mHOHHW ”Hmcgmum ”GM WQHMEHUWH USSHvanHH gaflunflz film mqmdfi 87 TABLE 5-8 Population Values of 01/2 and 81/2 2 ¢1/2 1/2 F3.034 (symmetric) 2.247 (symmetric)- I 0.0 0.776 0.0 2.247 0.0 0.0 2.349 0.0 0.0 2.247 L — L0.0 0.0 0.0 2.247 7 . 7' . _ 3.034 (symmetric) 2.247 (symmetric) II 0.149 0.762 0.0 2.247 -.092 1.397 1.886 0.0 0.0 2.247 7 J 0.0 0.0 0.0 2.247J _ r' _ 3.034 (symmetric) 2.247 (symmetric) 111 0.149 0.776 0.0 2.247 L_-.092 1.397 1.886 0.0 0.0 2.247 A L0.0 0.0 0.0 2.247 88 .2m 80 «>828: rem>ooza ra>oza H6808 moxaa How mGOHuQESmmm may now mcoflumfismmm map How mcoHumEdmmm mnu "monHom mm omcmmoo ou mumfiumonmm811HHH ou mumHHQOHQQMIIHH cu muMHHQOHQQMIIH mm m xfluumfi cucumum>oo 8:88 mmm.w mmw.w mew.m oww.w mew.w mme.m mmw.w mmH.w mem.m, ram wwm.w mem.e wmm.e mmm.w mwm.w mom.m mmm.w mmm.w wmw.m «me mmm. mwm. mmm. mmm. mmm. 8mm. mmm. mmm. ewm.w «me wme.w Hme.w. mem.: mmm.w mmm.a mmw.- mme.w mme.H mom.. 2.8 HHH ooe. wmm. mme. awe. mwe. eem. mee. mwe. mmm. wwo wmm. wmm. mwe. mam. mam. eoe. com. com. mme. awe mwm.w emm.w mom.m mmm.w mmm.w mmm.m wem.w mmm.w emm.m aae mww.w eew.w mme.m mww.H moa.H mem.e mww.w How.w mem.e 2am mmm.w mmm.w omw.m mmm.w mwm.w mmw.m mwm.w mwm.w owm.m ”we mmm. mmm. woo.a mmm. mmm. mmm. mmm. mmm. mmm. wme mme.w mme.w emw.- wem.w mmm.w mmw.- mme.w mme.w wmm.- ame HH moe. mmm. mwm. mme. mme. emm. eme. ome. wmm. wwe mmm. mmm. moe. mmm. mmm. mmm. mmm. wmm. awe. “we mmm.w mmm.w mem.m emm.w mem.w mmm.m emm.w mmm.w mmm.m 8 eHo.w mmm.w mwm.m mmm.w mmm.w wmm.m mmm.a wmm.w wmm.m 2am mmm.w mmm.w mmm.e mam.w mmm.w ooo.m mwm.w mem.w mmm.e ".8 H owm. mmm. mmw. cam. mme. ewe. mom. mme. moe. wwe mmm.w ewm.w mmm.m wem.w eem.w mmm.m mam.w mmm.w wem.m 228 .88 wmm aawz mmm wmm aawz mum wmm amoz umumsmnmm 8w muomwwm HHa uomumw a msmofi Hmsmm U mmmo m mmmu « mmmu mlm mqmda & can 0 How muounm Unmocmum can mmumsflumm oooaflmeHA fidfifixmz CHAPTER VI IMPLICATIONS OF RESULTS One of the main purposes of the present study is to form guidelines for researchers working with repeated meas— ures data from the results of the analysis procedures as applied to the simulated data reported in Chapter V. In a strict sense, generalizations from this type of study are limited to the design parameter values and sample size specified as well as to the underlying distribution which is assumed to be multivariate normal. And, although sta- tistical tests performed on the generation routine showed it to be satisfactory, it is also possible that the data do not have the intended properties. Nevertheless, conclusions can be drawn from the results which should be beneficial to the behavioral scientist. In this chapter, some possible guidelines for selecting one of the three analysis models will be discussed followed by ideas for further research to expand and substantiate the present conclusions. Guidelines for the Behavioral Scientist A successful research project is largely dependent on choosing a design and analysis procedure which speaks to 89 90 the research hypotheses generated from the objectives of the study. Assume the design that has been adopted for a particular study calls for the subjects to be measured on the same variables at different points in time or under different experimental conditions. A repeated measures analysis procedure would be appropriate. Mixed model ANOVA, MANOVA of RM, and ANCOVST are three possible choices for analyzing the data. The experimenter now needs to consider which of the three models best suits the given situation. I First consideration should be given to the hypoth- esis of interest. If one is investigating mean level dif- ferences across the occasions, any of the three models is potentially useful and further thought needs to be given to other issues before making the final choice. If the hypoth- esis concerns the structure of the covariance matrix, e.g. one wants to examine the latent sources of variation among subjects, ANCOVST is the only one of the three models which is applicable. A second question for an experimenter to ask when choosing an analysis pertains to the importance of commit- ting a Type I or Type II error when testing the repeated measures hypothesis. If one wants to guard against reject- ing the null when it is true, a model in which the a level is close to the theoretical value or one classified as a conservative test would be the best choice. When the 91 assumptions for mixed model ANOVA are met (case I-A), the empirical estimates of a are closest to the theoretical values for mixed model ANOVA than for the other two pro- cedures. In this same situation, MANOVA of RM and ANCOVST are more likely to reject the null hypothesis when it is true. If these assumptions, however, are not met, all three models lean toward a conservative test, with the exception of case II-A for ANCOVST. On the other hand, the researcher may want a test which is powerful enough to detect even small differences if they exist. ANCOVST would be a good choice as it is the most powerful of the three methods irregardless of the structure of 2. As the size of the effects increases, MANOVA of RM gains more power relative to ANCOVST in detecting the differences and thus is a good selection. Properties of the estimates of the repeated measures effects should also be an area of concern when choosing an analysis. The least squares estimates and the maximum like- lihood estimates of effects are identical in value. Both sets of estimates are unbiased. But the maximum likelihood estimates found in ANCOVST generally have a smaller standard error. If one plans a post hoc confidence interval proce- dure to determine which of the effects are nonzero, ANCOVST will give narrower confidence bands. If estimates for the variance components of the latent random variables are 92 necessary for the research questions, then, of the three models, ANCOVST is the best choice. A fourth point a researcher should consider is the number of dependent variables measured in the study. Often subjects are measured on more than one variable at each time point or experimental condition. MANOVA of RM should be given serious thought in this case as the hypothesis for the model includes repeated measures effects over all dependent variables simultaneously. This allows a control for the experimentwise error rate. Final consideration must be given to the availabil- ity of computer programs for a given analysis and the cost of running the program. Programs for all three procedures are available on the CDC 6500 at Michigan State University. MULTIVARIANCE (Scheifley and Schmidt, 1973) performs MANOVA of RM for factorial designs over subjects and repeated meas- ures and also prints the terms needed to collapse to the mixed model ANOVA if desired. If a researcher has selected the mixed model ANOVA, the PROFILE program (Wright, 1970) is also available. Besides the ANOVA table, output includes statistical tests for the assumptions of the model which can aid in choosing an appropriate analysis procedure. It does not estimate the repeated measures effects. PROFILE is also limited to one between groups factor and one repeated meas- ures factor. The program for ANCOVST is available from the 93 author although there is no documentation for it at this time. The expense involved in running each of these programs is different and varies with the size of the design and the number of subjects. To illustrate this, a set of data collected on 20 subjects was analyzed by each routine. The research design is a 32 factorial design over-the repeated measures. The results from PROFILE cost $0.12 while MULTIVARIANCE was $1.43. When ANCOVST was applied to this set of data, only the variance components for the main class model were estimated as the means were not part of the program at that time. The cost was $1.48 when the iterations began at an arbitrary point. This value decreased to $0.52 when initial estimates were used which fell close to the converged values. As the number of param- eters estimated in ANCOVST increases so also do the costs rapidly increase. When good approximations for parameters are available, the iteration procedure takes less time and, therefore, the cost decreases. PROFILE will usually be the least expensive analysis but the amount of information given from this routine is also less than from the other programs. The iterative procedure of ANCOVST can take a good deal of computer time, especially if the routine is not able to find a minimum point immediately. MULTIVARIANCE is often a good choice 94 as the amount of information available and the flexibility of the program make the expense worthwhile. Further Research The guidelines just discussed are based on a single sample 22 repeated measures design with N==30. One way to substantiate or refute the results is to carry out more studies of a similar nature with other designs over the repeated measures and varying sample sizes. For example, research studies often have fewer number of subjects with complete data at the end of the project than expected at the start. So expectations of N==30 may reduce to N==20. In this case the researcher must know if the above guidelines still apply. The design over the repeated measures may involve a time dimension and the interest is in fitting a polynomial equation to the data. Again, the question is, do the guidelines still apply? Adding a design over the subjects also complicates the analysis as there are now subject effects and subject x repeated measures interaction effects. Data do not always meet the assumptions of multi- variate normality employed in this study and it is also important to know the consequences of deviation from this distribution. The possible extension of the present study are many and will be of use in the behavioral sciences. Another possible area for research concerns the ANCOVST model. Its properties have not been thoroughly 95 explored, especially those pertaining to the estimates of 4 and Y. As was mentioned in Chapter V, exact estimates of the standard errors are available when equations for the second derivatives are known. Thus, adding these calcula- tions to the existing program will give more information about the parameter estimates. Another way to make the ANCOVST program more efficient is to investigate procedures for approximating the estimates to use as the initial start- ing points. The choice of an appropriate analysis for a repeated measures design is not always straightforward. The advan— tages and disadvantages of mixed model ANOVA, MANOVA of RM, and ANCOVST must be weighed carefully within the constraints of the specific situation and the facilities available. Further comparisons among these procedures will give addi- tional-aids in the decision making process. APPENDIX 10 15 20 25 30 35 b0 15 50 55 PROGRAM 0000000000000 96 GENDATA °‘ PROGRAM GENDATA(INPUT,OUTPUT.TAPE5=INPUT,TAPE6=OUTPUT,PUNCH) GENERATION PROGRAM FOR MY DISSERTATION. READ IN SIGMA AND MU AS DETERMINED AHEAD OF TIME. FIND CHOL 0F SIGMA = A GENERATE M RM DISTRIBUTED N(0,1) ' Z - 3 X 1 TRANSFORM Y = AZ 9 MU GENERATE N VECTORS, IoEo SUBJECTS CALCULATE MEAN AND COV MATRIX SUDROUTINES NEEDED CHAMS CHOL DIMENSION H116),HU41),SIGHA(10),A(~,~),IEHP(~,~),xaARtu,1) DIMENSION Z(4,1),v(300,41,rxu,«),1(10).s41c1 REAL nu ‘ ser GENERATOR PARAMETERS A1 - 3.9h9886138 A3 = .252h0878h A5 = OJ?65“2912 A7 = .008355968 A9 = .029899776 IX ' 135791357 READ IN NO. OF POPULATIONS READC5,3)NPOP 3 FORMATCIS) 00 100 IJK = 1,NPOP IJKK = IJK 9 5 HRIT£(6,5)IJK,NPOP 5 FORMATC‘1FOPULATION NO.’,I5,' OUT OF 'gIS) READ IN HEADING, K, N, NSAM, AND ICARO REAO(5,1)H,K,N,NSAM 1 FORMAI(8A10/8A10/AIS) NDIM = K ’ (K t 1)/2 WRITE(6,2)H,K,N,NSAM,NDIM 2 F0?MAT(2('0’,8A10/),’ K = NO. OF RM’,IDI' N = NO. OF SUBJ‘.IBI A' NSAH : NO. OF SAMPLES GENERATEO’.IBI 0' NDIH = NO. OF ELEMENTS IN SIGHA',IB) XX = N READ IN MU AND SIGMA PEAO(S,101)(MU(I),I=1,K) 101 FORMAT(1CF6.0) HRITE(69201’(HU(I,9I=19K1 201 FDRMAT(’CTHE MEAN VECTOR FOR TRANSFORMATION’I/IO(12X,F10.3/)1 READ(5,101)(SIGMA(I),I=1,NDIM) HRITE(6,2021 202 FORMAT(’O THE COVARIANCE MATRIX FOR TRANSFORMATION’II) HRITE(6,203)(SIGMA(I),I=1,NDIM) 203 FCRMAT(5X,F10.3/5X,2F10.3/5X,3F10.3/5X,8F10.3) FIND CHOL 0F SIGMA = A CALL CH0L(SIGHA,A9K,DET) CALL CHAHS‘Agtpr3pA’ HHITEC6920QT 20h FORMATT’OTHE CHOLESKY 0F SIGMA = T’II) 97 PROGRAM GENDATA DO 110 I = 19K HRITE(6,205)(TTIgJ19J=1,K) 205 FORMAT(2X.5F10.“) 110 CONTINUE 60 HRITEl6,206) 206 FORMATT'OSAMPLE',12X,'SAMPLE MEANS'.J§X,’SAMPLE COVARIANCE MATRIX' Al’ NUMBER'//) 0 BEGIN LOOP FOR GENERATION OF M VARS FOR N SUBJECTS DO 1000 LOOP = igNSAM 65 DO 10 II = 1gN DO 20 J = 1gK 8 = 0 DO 12 KK= 1,12 IV = IX ’ 16777219 70 IX = IV IF(IY)7.8,8 7 IV = IV 0 281k7h97671065501 0 YFL = IY YFL = YFL ’ .355276-14 75 0=OoYFL 12 CONTINUE R 3 (U ‘ 607’“. RS 3 R ‘ R ZTJpIT = ((((A9 ’RS 0 ATT’RS O AST’RS 0 AJT‘RS O AIT‘R 80 20 CONTINUE C FIND Z = T T Z O MU - TRANSFORMED OBSERVATION C HULTIPLY ROUTINE DO 183 I = 1,1 7 DO 1N3 J = 19K 85 X = 00 00 IRA KK 3 19K . X = X f T‘J'KKT T ZTKK’IT ' . IHI CONTINUE TENP‘JgI) = X 90 1ND CONTINUE C A00 ROUTINE DO IHZ I 3 19K ZIIol) = TEHPII’IT O HU‘I) . I“? CONTINUE 95 00 3C J ‘ VIII’JT 30 CONTINUE 10 CONTINUE -19K ZTin) . C CALCULATE MEAN AND VARIANCE 100 00 40 J = 1.x XBARtJ) = 0 00 «c I = 1,1 XDAR(J) = XBAR(J) 4 741.4) 90 CONTINUE 105 ,- 00 41 J = 1.1 - ‘41 XBAR(J) = XBAR(J)/N C SUBROUTINE GRAM IC = 0 _ .. 00 153 I = 1.x 11o ' 00 150 J = 1.x 115 120 125 130 135 '1h0 PROGRAM 98 GENDATA IC = IC 0 1 HTIC) = XDARTIpI)’X8AR(J,1) 150 CONTINUE H IS A K X K SYMMETRIC MATRIX - XBAR X XBAR TRANS DD 152 I = 19NDIM 152 HTI) = HTI) ‘ N SUBROUTINE GRAMT IC = 0 DO 153 I DO 153 J X = 00 00 15h KK = 19N X = X 0 Y‘KK’IT ’ YTKK,JT 15k CONTINUE IC = 10 0 1 S(IC) = X 153 CONTINUE 5 IS A K X K SYMMETRIC MATRIX - Y TRANS X Y 00 155 I = 1,NDIM $11) = 5(1) - H(I) 155 CONTINUE DO 90 I = 1.NDIM 5(1) = STI)IXX 90 CONTINUE . PUNCH 212,(X8ARTI).I=1,K),(S(I),I=1,NDIM).IJKK 212 F0RHAT(‘OF502’F502’9F6029117 IF(LCOP.GT.10.ANO.LOOP.LT.9901GO TO 1000 . HRITET6,21Q)LOOP,(X8AR(I),I=1,K),(STIT,I=1,NDIM) 214 F02MATT2X,IA,3X,AF7.2.3X,10F7.2) 100C CONTINUE 100 CONTINUE END 1,K 191 10 15 20 25 30 SUBROUTINE CHAMS 00000000000000 0 SUDROUTINE CMAMS(A,8,N,MA,M81 CMAMS SUBROUTINE A1 A3 99 CALL CHAMSTA,8,N,MSA,MSBT 3320) CONVERT AN N BY N MATRIX STORED IN MODE MA MATRIX TO BE ARRAY HHERE CONVERTED MATRIX HILL 8E STORED DIMENSION OF MATRIX A MODE OF STORAGE OF A 8 MODE OF STORAGE OF 8 CONVERTED MATRIX STORED IN MODE M8 8 MAY REPLACE A L = N 4 1 K = (L’NTIZ 0 1 00 ‘01 J:19N JR = L ' DO “1 I:19JR IR = JR - K = K - BTJRgIRT DO “3 J=2.N L = J - 1 1 J I O 1 = A‘Kng DO “3 I=19L B‘I’JT END 0 DIMENSION ATN,1),8(N,1) LOWER TRUE TRIANGLE T0 SQUARE TO AN N BY 100 SUDROUTINE CHOL SUBROUTINE CMOLTA,8,N,C) ‘ CALL CHOLTAgflgN,C) AN N BY N SYMMETRIC MATRIX AN ARRAY OF AT LEAST N’TNOlT/Z LOCATIONS NUMBER OF ROHS IN A A RETURN CELL FOR THE DETERMINANT 0F 8 CALCULATE THE CHOLESKY FACTCR OF A AND STORE IN 8 STORE THE PRODUCT OF THE DIAGONAL ELEMENTS 0F 8 IN C IN TRUE TRIANGULAR FORM 000000000000 020D DIMENSION A11),B(1) X = SORTIATIT) C=X 8(1)=X N1=N'1 KC=1 IFIR=1 DO 101 J=10N1 KC=KCtJ 101 8(KC)=A(KCTIX DO 100 I:1,N1 IFIR=IFIR+I KC=IFIR X=0. DO 102 J=1,I X=XOG(KC)"2 102 KC£KC91 X= SOQT(A(KC)-X) C=C'x ' . IHKC)=X - II=If1 IF (II.EO.N) RETURN JC=IFIR ‘00 103 J=II9N1 JC=JCOJ IC=JC KC=IFIR '31). DO 13“ K3191 Y=YOB(IC1’8(KC) KC=KCO1 10b IC=IC91 103 BTIC)=(A(IC)-Y)/X 100 CONTINUE RETURN END BIBLIOGRAPHY BIBLIOGRAPHY Anderson, Theodore W. ‘An Introduction to Multivariate Statistical Analysis. New York: Wiley and Sons, Inc., 1958. Book, R. D. Components of variance analysis as a struc- tural and discriminal analysis for psychological tests. British Journal of Statistical Psychology, 1960, 13, 151-163. Bock, R. D., and Bergmann, R. E. Analysis of covariance structures. Ps chometrik , 1966, 31, 507-533. Bock, R. D., and Haggard. The use of multivariate analysis of variance in behavioral research. In D. K. Whitla (Ed.), Handbook of Measurement and Assessment in Behavioraleciences. Mass.: Addison-Wesley, 1968. Bock, R. D., and Peterson, A. Matrix operation subroutines for statistical computation. University of Chicago, 1967. Cole, J. W. L., and Grizzle, J. E. Applications of multi— variate analysis of variance to repeated measurements experiments. Biometrics, 1966, 22, 810-828. Collier, Raymond 0., Baker, Frank B., Mandville, Garrett K., and Hayes, Thomas F. Estimates of test size for several test procedures based on conventional variance ratio in the repeated measures design. Psychometrika, 1967, 32, 339-353. David, F. N., and Johnson, N. L. The effect of non- normality on the power function of the F test in the analysis of variance. Biometrika, 38, 1951, 43-57. Danford, M. B., Hughes, Harry M., and McNee, R. C. On the analysis of repeated measures experiments. Biometrics, Davidson, Michael L. Univariate versus multivariate tests in repeated measures experiments. Psychological Bulle- tin, 1972, 77, 446-452. 101 102 Donaldson, Theodore S. Robustness of the F test to errors of both kinds and the correlation between the numerator and denominator of the F ratio. Journal of the American Statistical Association, 1968, 63, 660-676. Elston, R. C., and Grizzle, J. E. Estimation of time- response curves and their confidence bands. Biometrics, 1962, 18, 148-159. Finn, Jeremy. Multivariate analysis of repeated measures data. Multivariate Behavioral Research, 1969, 4, 391-413. Fletcher, R., and Powell, M. J. D. A rapidly convergent descent method for minimization. Computer Journal, 1963, 2, 163-168. Glass, Gene V., Peckham, Percy D., and Sanders, James R. Consequences of failure to meet assumptions underlying the fixed effects analyses of variance and covariance. Review of Educational Research, 1972, 42, 237-288. Gordon, Geoffrey. System Simulation. New Jersey: Prentice- Hall, Inc., 1969. Greenhouse, Samuel W., and Geisser, Seymour. On methods in the analysis of profile data. Psychometrika, 1959, 24, 95-112. Hummel, Thomas J., and Sligo, Joseph B. Empirical compar- isons of univariate and multivariate analysis of variance procedures. Psychological Bulletin, 1971, 76, 49-57. Jansson, Birger. Random Number Generators. Stockholm, 1966. Joreskog, K. G. A general method for analysis of covariance structures with application. Educational Testing Service Research Bulletin 69-46, 1969. Joreskog, K. G. A general method for analysis of covariance structures. Biometrika, 1970, 57, 239-251. Joreskog, K. G. A general approach to confirmatory maximum likelihood factor analysis. Psychometrika, 1969, 34, 182-202. Knuth, Donald E. Seminumerical Algorithms: The Art of Computer Programming. Mass.: Addison-Wesley Publishing Co., 1968. 103 Lana, Robert E., and Lubin, Ardie. The effect of correlation on the repeated measures design. Educational and Psychological Measurement, 1963, 23, 729-739. Miller, D. M., and Lutz, M. V. Item design for an inventory of teaching practices and learning situations. Journal of Educational Measurement, 1966, 3, 53-61. Potthoff, Richard F., and Roy, S. N. A generalized multi- variate analysis of variance model useful especially for growth curve problems. Biometrika, 1964, 51, 313-326. Rao, C. R. Linear Statistical Inference and Its Applicg: tions. New York: Wiley and Sons, Inc., 1965. Roy, J. Step-down procedures in multivariate analysis. Annals of Mathematical Statistics, 1954, 29, 1177-1187. Scheifley, V. M., and Schmidt, W. H. Jeremy D. Finn's multivariance--univariate and multivariate analysis of variance, covariance, and regression. Occasional Paper No. 22, Office of Research Consultation, College of Education, Michigan State University, 1973. Schmidt, William H. Use of the analysis of covariance structures to explore the issue of uncorrelated latent random variables. Paper presented at the annual meeting of the American Educational Research Association, 1972. Schmidt, William H. Matrix operations subroutines for statistical computation--an adaption of the matrix package of R. D. Book and Allan Peterson for use on the Michigan State University CDC 6500. Occasional Paper No. 13, Office of Research Consultation, College of Education, Michigan State University, 1970. Schmidt, William H., and Scheifley, Verda. Estimating the covariance matrix of the random components in the mixed model appropriate to the single sample repeated measures data. AmericanyEducational Research Journal, 1973, 10, 125-135. Schmidt, William H., and Wiley, David E. Simultaneous structural analysis of mean vectors and covariance matrices-~some applications to repeated measures data. Paper read at a meeting of the American Educational Research Association, 1971. 104 Wiley, David E., Schmidt, William H., and Bramble, William J. Studies of a class of covariance structure models. Journal of American Statistical Association, 1973, 68, 317-323. Wright, David J. PROFILE--A FORTRAN IV program for the analysis of split-plot factorial or groups by repeated measures designs. Occasional Paper No. 12, Office of Research Consultation, College of Education, Michigan State University, 1970. MICHIGAN STATE UNIV. LIBRARIES WWI”WW”WWHIHIIHIIIHHIIWWI 31293106665148