IL ‘I alt’ I V $.23. 533.3. J: .2. , 33...... ‘ 3......) ....r fin... i . 3.; . (.h :26 .1 .. ...h. .J.. . «3...: 33;. 1 p h fix? influx»; _ . 5w. . . ‘ f i; u... u its». rm‘ 1 r. ;9 Q . Ti a . 32. 1.90 :5. ..!.u.. 1: {.6 1 :1.» . . Tr...=!z4.l v s‘ \ e .. Susi: ., , z: a » , .. p .. E. “Aha“? , _ V 3 4‘ .z. h J.) . 5. 3.1.2.? :39, f: 321:1... Oll- yl .« .5112: ZS}. 11:411.. ‘ 7.2 110.». S . ' C ”v! . fit l.I\.o {SI-ll: . .OI .1..IJA..i, 7...: .ivnnu4v; :I‘nfln 1 .v .I .9... ‘ t. b.7‘ .4; l THESIS IlllllllllIllllllHIHHl.lllllllllllllllllllllllllllllllllll 301410 1848 This is to certify that the dissertation entitled A COMPARISON OF ME I'HODS FOR CORRECTING MULTIVARIATE DATA FOR ATTENUATION WITH APPLICATION TO SYNTHESIZING CORRELATION MATRICES presented by Christine M. Schram has been accepted towards fulfillment of the requirements for Ph.D. degmin CEPSE fiafiwma ajor professor Date M ‘25:) /C’79JJ V d MS U is an Affirmative Action/Equal Opportunity Institution 0. 12771 LIBRARY Michigan State University PLACE II RETURN BOX to remove We checkout from your record. TO AVOID FINES return on or before dete due. DATE DUE DATE DUE DATE DUE MSU le An Afflmetive Action/Emu Opportunity Imam m ”39.1 A COMPARISON OF METHODS FOR CORRECTING MULTIVARIATE DATA FOR ATTENUATION WITH APPLICATION TO SYNTHESIZING CORRELATION MATRICES BY Christine M. Schram A DISSERTATION Submitted to Michigan State University in partial fulfillment of the requirements for the degree of DOCTOR OF PHILOSOPHY Department of Counseling, Educational Psychology, and Special Education 1995 A COMPAE mF‘"\W ' ' '- .. Ocue‘v- All C0 ‘ rquat CH ABSTRACT A COMPARISON OF METHODS FOR CORRECTING MULTIVARIATE DATA FOR ATTENUATION WITH APPLICATION TO SYNTHESIZING CORRELATION MATRICES BY Christine M. Schram Corrections for attenuation have long been used to adjust sample correlations for measurement error. Current research synthesis (meta-analysis) procedures involve synthesizing correlational data. The synthesis of multivariate correlation data raises several statistical questions including correcting for measurement error. The focus of this work was to discover the most statistically sound method for correcting multivariate data for attenuation which accounts for dependence among the correlations and reliabilities. Multivariate corrections from "errors-in-variables" regression analysis were examined, as was an existing multivariate correction from educational literature. These methods were compared to the traditional univariate correction for attenuation. Simulated and exact comparisons were made of corrected correlations and their resulting variance-covariance matrices. All the methods examined produced similar corrected correlations. Even the simple univariate correction yielded corrected correlations that were good estimates of the population correlation. In addition, a variety of COIIECC GLSSEIC d;str;b 1 tne or Similar on raw Syntnes approximations to the population variance-covariance matrices of the corrected correlations were associated with these correction methods. A variance estimator derived in this dissertation and based on large-sample theory yielded the best estimates of variation when.compared.to thelempirical sampling distribution. A related variance estimation method based on the correction from Fuller and Hidiroglou (1978) also gave similar results to the large—sample theory method, but relied on raw data, which are often unavailable in most research syntheses. An example illustrated the results of a multivariate synthesis using the new procedure. The results of this example showed more variability in the average correlations and larger' homogeneity' test statistics when compared to previous analyses of the same studies. Overall, if corrections are to be applied, the univariate correction, used with the large-sample variance-covariance matrix, will yield reasonable results. worthy : best she ehvka r—c if ‘ \ ‘nYfirur-Vy‘ Vitoduj‘. h:§~‘-. " ‘1“*$¢G.. Process fl ' ‘ uenmann’ Acknowledgements Now that this is over, I’m not sure I can produce words worthy of the gratitude I feel. I’m finding it difficult to appropriately thank those whose assistance and encouragement accompanied me on this journey. However, I will give it my best shot. I wish to thank my committee members for helping me get through this with ease. Dr. William Schmidt and Dr. Dennis Gilliland were always cooperative and available and made this process much easier than I expected. Thanks too to Dr. Irv Lehmann, for filling in admirably, for always being encouraging, and for always having time to listen. My undying gratitude goes out to my advisor and mentor, Dr. Betsy Becker, who not only has been supportive and essential to my progress, but is a great role model, and most of all, a great friend. I also wish to thank those who were influential early in my academic career, especially Dr. William Mehrens and Dr. Michael Seltzer who epitomized class whenever I worked with them. Finally, I have to thank my family and friends for their support and their attempts to understand when they knew they iv couldn’t keeping v couldn’t possibly. Thanks Alex, the love of my life, for keeping me sane, making me laugh, and making the sacrifices more bearable. Maybe now we can go on a vacation. TABLE OF CONTENTS LIST OF TABLES ............................................ x CHAPTER 1: INTRODUCTION .................................. 1 Meta-analysis in Educational Research .................. 1 Purpose of the Study ................................... 2 Research Questions ..................................... 4 Overview of the Dissertation ........................... 8 Summary ................................................ 9 CHAPTER 2: LITERATURE REVIEW ............................ 10 Measurement Error ..................................... 10 Multivariate Meta-analysis ............................ 13 Harris and Rosenthal .............................. 13 Schmidt, Hunter, and Outerbridge .................. 14 Premack and Hunter ................................ 15 Becker and Schram ................................. 15 Validity Generalization and Corrections ............... 17 Other Univariate Corrections .......................... 20 Hakstian, Schroeder, and Rogers ................... 20 Meta-analysis and corrections ..................... 21 Measurement Problems in Univariate Syntheses .......... 22 Measurement Problems in Multivariate Syntheses ........ 24 vi Q ‘ Ex' Fr PU «E 8.... Sta Ti «.2 vii Existing Multivariate Corrections ..................... 25 Fuller and Hidiroglou ............................... 25 Gleser .............................................. 26 Book and Petersen ................................... 27 Summary ............................................... 28 CHAPTER 3: METHODOLOGY .................................. 30 Statistical Formulations .............................. 3O Notation ........................................... 30 Multivariate Analysis .......... ' .................... 31 Distributions of Corrected and Uncorrected Correlations ....................................... 32 Variances ....................................... 32 Covariances ..................................... 32 Layout of Simulation .................................. 34 Data Generation .................................... 34 Different Univariate Corrections ................... 36 Simulation Parameters .............................. 37 Sample sizes .................................... 37 Correlations .................................... 38 Reliabilities ................................... 4O Basis for Comparing Methods of Corrections ......... 41 Corrected correlations .......................... 41 Variance-covariance matrices .................... 42 Comparisons Made ...................................... 43 Exact Comparisons ............................... 43 Simulation Comparisons .......................... 44 Methods for the Example ......................... 44 Exac Tl ab Assn Simu PC viii CHAPTER 4: RESULTS ...................................... 46 Exact Comparisons among the Methods ................... 46 Gleser ............................................. 46 Bock and Petersen .................................. 48 Fuller and Hidiroglou .............................. 49 The Three Variable Case ............................ 51 Summary of Exact Results ........................... 52 Assumptions of each Method ............................ 53 Simulation Results .................................... 55 Caveats ............................................ 55 Results from Count Data ............................ 56 Corrected correlations greater than unity ....... 56 Determinants of the corrected correlation matrices ........................................ 66 Determinants of variance-covariance matrices of the corrected correlations ................... 76 The size of the determinants of the variance- covariance matrices ............................. 85 Summary of counts ............................... 94 Results of Magnitude Data .......................... 97 Corrected correlations .......................... 97 Variances ...................................... 105 Covariances .................................... 122 An Application of the Methods to Existing Data....122 Testing the homogeneity of the correlation matrices ....................................... 123 Estimating a common correlation matrix ......... 125 Results from the example ....................... 125 APPEJ“ U11 RELI} ix CHAPTER 5: SUMMARY AND CONCLUSIONS ..................... 128 Results .............................................. 128 Other Findings ....................................... 133 Applications of this Work ............................ 133 Further Investigations ............................... 134 Conclusions .......................................... 135 APPENDIX A: VARIANCES AND COVARIANCES OF CORRECTED CORRELATIONS ......................................... 137 APPENDIX B: COVARIANCES AMONG CORRELATIONS (INCLUDING RELIABILITIES) ....................................... 138 APPENDIX C: METHODS USED FOR CORRECTING FOR ATTENUATION ..................................................... 139 APPENDIX D: FORTRAN PROGRAM USED IN SIMULATION ......... 141 REFERENCES .............................................. 155 18 Q Tab ab-e Y a tha Fl. Table Cova Table Matr: T. .v .. .«k ahu ”Nu D . Table l LIST OF TABLES Table 1: Simulation Parameters ......................... 39 Table 2: Percent of Corrected Correlations Greater than Unity ............................................. 58 Table 3: Percent of Determinants of Corrected Correlation Matrices less than or equal to Zero ........ 67 Table 4: Percent of Determinants of Variance- Covariance Matrices less than or equal to Zero ......... 77 Table 5: Mean Determinants of Variance-Covariance Matrices of the Corrected Correlations ................. 87 Table 6: Number of Times per Case where the Fuller and Hidiroglou and Book and Petersen Adjustments were needed ................................................. 96 Table 7: Mean Corrected Correlations ................... 98 Table 8: Mean Variances of Corrected Correlations ..... 106 Table 9: Mean Covariances of Corrected Correlations...115 Table 10: Methods and Results from Example Data ........ 127 CHAPTER I INTRODUCTION Meta-analysis in Educational Research Meta-analysis is a developing statistical technique that allows for the synthesis of results from studies of the same phenomenon. Such studies generally contain data sufficient to compute an effect size, which exhibits the magnitude of the relationship studied. Meta-analytic statistical techniques are then used to combine effect sizes, and summarize the results. Further analysis allows the explanation of the variability in effect sizes by the modeling of moderating variables. Meta-analyses have been criticized for being too simplistic and for lacking in theory (Chow, 1987). Because many meta—analyses simply summarize bivariate relationships without accounting for moderators, or other significant relationships, this criticism is justified. One potential answer to this criticism is to develop methodology which will allow the synthesis of more complicated theory-based data. Meta-analytic techniques are currently being developed which allow the summary of multivariate relationships (see, e.g., Becker & Fahrbach, 1994,- Becker & Schram, 1993). These techniques allow the synthesis of interrelationships (correlations) among variables, in contrast to the current syntheses of simple bivariate relationships. Resulting synthese of educa Mul (models : synthese: However , leads to 2 syntheses will allow for a more complex and complete picture of educational phenomena. Multivariate syntheses examine systems of variables (models of interrelated variables), in contrast to univariate syntheses, which summarize data about one relationship. However, synthesizing models by combining correlation matrices leads to several statistical problems, one of which involves correcting for attenuation in a multivariate setting. The multivariate nature of such corrections led to several research questions. This work examined the effects of using various corrections and asked whether there is one best multivariate correction for attenuation, for the particular case of multivariate research syntheses. Purpose of the Study The purpose of this work was to investigate the possibilities of using multivariate corrections for attenuation. Is there a best possible correction? How do corrections proposed in the "errors-in-variables" regression literature apply? The meta—analytic context differs from the regression context in that reliabilities in educational and social-science data are often scarce or unavailabLe. What assumptions are necessary before these corrections can be used? Wherever multivariate situations arise when the assumptions are met, these techniques should be applicable, .and.in fact, the applications reach beyond meta-analysis. Because populati 3 The resultsrof this‘work.are important because correcting for attenuation will provide better estimates of parameters. Because sample correlation coefficients underestimate population parameters, correcting for this measurement error will yield.more accurate results in research syntheses. .Also, the power of statistical tests is decreased if correlations are uncorrected (Williams & Zimmerman, 1982). If the methodology exists for getting better estimates, better analyses (including syntheses) will result. Certainly, the usefulness of getting the best possible estimates is obvious. We want the analyses to be as "right" as possible. Four factors contribute to the need for further study of corrections in multivariate analyses: 1) measurement errors affect correlation coefficients, 2) dependence in multivariate data can lead to inaccurate analyses, 3) measurement errors may be correlated and 4) corrected correlations have different variances and covariances than uncorrected correlations. However, simply correcting each correlation in.a matrix using traditional univariate corrections can lead to problems. Bock and Petersen (1975) noted that the resulting variance- covariance matrix associated with the correlation coefficients may not be positive definite or positive semidefinite. Preliminary simulation results showed that this possibility exists. There are several techniques which could be used to exorrect correlation matrices, and ultimately their variances and cova through e provide correlat; such corr the most 4 and covariances, for attenuation. Applying these approaches will potentially yield different results. Contrasting them through exact work (derivations) and through simulation should provide useful information about the best way to correct correlation matrices. The use of a sample data set shows how such corrections can influence results in.one case. .Selecting the most appropriate and applicable procedures for meta- analytic situations is the focus of this work. Research Questions Several questions can be raised about multivariate attenuation corrections. The questions addressed in this study are listed below, with a brief description of each problem, a description of how each was investigated, and the anticipated results. 1. What are the consequences of using a simple univariate correction for each of a set of correlations? The univariate correction ignores the dependence in the data, so problems with this approach are expected. These problems may take the form of out-of—range corrected correlations (values greater than one), or correlation matrices and variance- covariance ‘matrices for' the correlations *which. are non- positive definite. If the usual univariate correction-for-attenuation formulas are used, and the corrected correlations are substituted into the formula for the variance of a correlation coefficient, is the result acceptable? This first research question their vai that th: variances distribut covarianc above), v large-Ban c - . I. reliar 5 question was answered by simulating correlations and computing their variances and examining the results. It was expected that this correction would give significantly smaller variances than those found by examining the sampling distributions of disattenuated correlations. 2. What would be the difference in variances and covariances based on the univariate correction (mentioned above) , versus using a variance-covariance matrix derived from large-sample distribution theory for correlation coefficients? If reliabilities are correlation coefficients, then corrected correlations are functions of correlations. As such, their large-sample distributions can be derived using results found in Olkin and Siotani (1976). The resulting variance- covariance matrix will take into account the covariances between.the reliabilities and the sample correlations, and the covariances among the reliabilities. This method treats reliabilities as random variables rather than fixed, population quantities as is assumed by other methods mentioned below. This method was compared to the variances and covariances associated with the univariate correction discussed in question #1, through both exact work and simulations. Exact work showed how the variances and covariances from these methods differed, and showed that the large-sample method based on Olkin and Siotani gave larger variances and covarian sample c coeffi 11 for raw How do t1 correctic from Book 6 covariances than the univariate method because the large- sample correction accounts for variability in reliability coefficients. 3. Several multivariate attenuation corrections exist for raw data, including many in the regression literature. How do these corrections compare to one another, and to the corrections mentioned above? The nmltivariate corrections frtmiBock:& Petersen (1975), Fuller and.Hidiroglou (1978), and Gleser (1992) were compared to one another, and to the corrections already discussed” The assumptions necessary for the use of each method were examined, to see if the assumptions are met in multivariate syntheses. The articles by Fuller and. Hidiroglou (1978) and, Gleser (1992) give corrections used in regression. These formulas were examined to determine whether the corrections can_ be applied to multivariate synthesis, and whether they can be compared to corrections based on classical test theory, for example, the correction of Bock and Petersen (1975). This investigation determined. whether any' or all of these corrections are appropriate for multivariate syntheses, and whether the corrections give acceptable results. 4. Which correction is most feasible and provides the best results? The corrections were compared on several bases. First, the frequencies of out-of—range corrected correlations and non-positive definite variance-covariance matrices for correlations were noted. The percentage of out-of—range assumpti: possibil; syntheses Exac correctic the corre CorreCtiC that is 1186. 7 values was recorded for each method and each simulation situation. Second, the variances and covariances of corrected correlations to expected values were compared, and the degree of bias in each was assessed. Finally, the assumptions necessary for the use of each correction, and the possibility of meeting these assumptions in multivariate syntheses were discussed. Exact work was used when possible to show how the corrections differ in theory. A simulation study showed how the corrections behaved in applications and in theory. The results of this investigation suggest the best method of correcting multivariate data for attenuation. The best correction is the one with the best statistical properties that is feasible in terms of assumptions necessary for its use. 5. How do these corrections affect results of multivariate syntheses? A set of data was examined using the methods recommended by the results of this work. The applications of the corrections to this data set illustrated the different methods in a practical setting, and focused on the consequences of using the corrections described. This data was from Schmidt, Hunter and Outerbridge (1986) and consisted of correlations representing relationships among five 'variables, and. their' jpopulation. and. estimated reliabilities. need to correlate distr:ou:( d;str1but literatur. Should re espemall} ”SW can c< (‘37 e op 8 Future directions for research on multivariate corrections were also addressed. Additional questions which need to be answered concern reliability distributions and correlated measurement errors. How are reliabilities distributed? Hypothetical, "assumed" reliability distributions have been used in the validity-generalization literature. Are these distributions accurate and appropriate? Should reliability in both the predictor and criterion be considered? Can correcting for both lead to further problems, especially if the reliabilities themselves are correlated? How can correlated measurement errors be estimated and what are appropriate values for such errors? These questions were not answered in this work, but seem critical for future research, thus, are discussed extensively in the final chapter of this dissertation. Overview of the Dissertation The dissertation contains four additional chapters. The second chapter addresses the review of the literature including the basis for corrections, their current application in multivariate syntheses, and a description of existing multivariate corrections. The third chapter details the methods used in this work. The fourth chapter summarizes the results, and the fifth chapter discusses recommendations and future research directions. multivar; using the Summary This work investigated the statistical and practical effects of multivariate corrections for attenuation. Different methods for correcting for attenuation in multivariate situations were described, and the effects of using these different corrections and the problems arising from each were examined. Potential problems included non- positive definite covariance matrices, and out-of—range variances and correlations. Practical aspects of corrections were illustrated using a multivariate synthesis example, and the ramifications of the different corrections were discussed. The results of this work include a justification and explanation of the most useful correction(s) for use in meta- analytic syntheses. 'This investigation. showed. that the univariate correction was a special case of each method. However, the method from.Fuller and.Hidiroglou (1978), and the method derived from large-sample theory yielded variances which most closely fit the sampling distributions in the simulation. Evidence is presented to show why these corrections are the best, and what problems exist with other, less useful corrections. Also, an indication of future research directions is presented. Syn( interreli methodolc between (Becker, and can 1 severely the best 1 Smtheses with maki SYcheses PErtinth meaSUremeI Validity E the data a CHAPTER II LITERATURE REVIEW Synthesizing data on models of multiple interrelationships is complicated. Current meta-analysis methodology has examined the synthesis of correlation matrices between variables assumed to be measured without error (Becker, 1992). However, this assumption is not warranted, and can be problematic since measurement error can produce severely underestimated correlation coefficients. However, the best way to correct for measurement error in multivariate syntheses is unknown. Several statistical problems associated with making corrections for attenuation have arisen in such syntheses and are discussed in this chapter. The literature pertinent to this problem covers several tOpics, including measurement error, multivariate analysis, meta-analysis and validity generalization, and statistical methods needed for the data analysis. This chapter details literature relevant to addressing the problems outlined in the first chapter. MW Whenever correlation coefficients are used, issues of measurement error arisen ZBecause of unreliability in.both.the loredictor and criterion, observed correlations underrepresent the true (p0pulation) correlation. Formulas for correcting for unreliability date back to Spearman (1904). The basic 10 formula correla: correlat reliabil- 11 formula for the correction for attenuation in the sample correlation._r_;xy is p'xy=rxy/1/2, (2.1) where p'xyiesthe estimated population correlation.between the fully reliable constructs x and y, r_ is the sample XY correlation, and pxx and pW are the known (population) reliabilities of the predictor and criterion measures respectively (Allen & Yen, 1979). The correlation p’xy in (2.1) is the maximum sample correlation which could be obtained, if no measurement error were present. Classical test theory forms the basis for this correction, based on the following assumptions: 1. X = T + E. An observed score is the sum of two parts, true score and error. 2. E(X) = T. The expected value of an observed score is the true score. 3. pET = 0. There is no correlation between error and true score for a population of examinees on one test. 4. p8182 = 0. The errors from two different tests are uncorrelated. 5. pElm = 0. The error on one test is uncorrelated with the true score on another test. 6. If two tests have observed scores X and X' that satisfy .Assumptions 1 through 5, and if for every population of examinees T=T' and 0E2 = ag.2 , the tests are called parallel tests. 7. If two tests have observed scores X1 and X2 that satisfy .Assumptions 1 through 5, and if for every population of examinees, T1 = T2 + c12 where c12 is constant, than the tests are called essentially tau- equivalent tests (Allen & Yen, 1979, p. 57). Giv among tr the has; reliabil (observe following above: pxx - where pxx Scores, a the Varia ObSe Either SC( reteSt r6 (aICErnat( an 3:111:6qu 12 Given these assumptions, one can derive relationships among true, error, and observed score variances, which form the basis for one definition of reliability. Theoretically, reliability is the ratio of true score variance to total (observed) score variance. For any group of test takers, the following relationships can be derived from the assumptions above: Pxx = 02.1. = 02x - 02E = 1 - 02E , (2.2) a x 02x 0’x 2 where pxx is the reliability, 0 T is the variance of the true scores, 0'2x is the variance of the observed scores, anddzE is the variance of the errors. Observed reliabilities can, be correlations, between either scores from two administrations of the same test (test- retest reliability) or scores on two versions of a test (alternate forms). Also, reliability can be obtained through an internal consistency measure based on one administration of a test. Test-retest and alternate-forms reliabilities are likely to contain errors in measurement that are not included in the observed correlation between variables at any single time point (e.g., errors due to change over time, practice effects, fatigue, differences in forms, etc.). IHowever, test— retest reliabilities are preferable to the others for correcting for attenuation, according to Lord and Novick (1968, p. 135). If the period before retesting is short, and fatigue factors are minimized, the other minimal errors in measurerrs 13 measurement are likely to lead to high estimates of reliability, which will, in turn, lead to conservatively corrected correlations, according to Lord and Novick. Alternate-forms reliabilities include additional error due to differences between forms, so this type of reliability is less useful for disattenuating corrections than test-retest. Lord and Novick also claimed that internal-consistency estimates can seriously underrepresent reliabilities, especially when there is a lack of item homogeneity. Thus, the use of internal consistency reliability for attenuation corrections is discouraged. The focus of this work will be on test-retest reliabilities, and both the exact work and simulations will consider this type of reliability and its assumptions. Multivariate Meta-analysis Multivariate syntheses are one type of analysis where multivariate corrections for attenuation are potentially useful» Four recent multivariate syntheses, Harris and Rosenthal (1985), Schmidt, Hunter and Outerbridge (1986), Premack and Hunter (1988), and Becker and Schram (1993) illustrate how and where these corrections could be applied. Harris and Rosenthal. Harris and Rosenthal (1985) were some of the first researchers to synthesize several relationships within one meta-analysis. They examined the literature on interpersonal expectancy effects using eight Ixnivariate meta-analyses. No attempt was made to investigate all.of the paths of their model simultaneously. They did not model de path deg which re correcti: in the te Spy, \vbb- examined job know ij ability, lOb perfc uilitary Studies. 14 model dependencies between paths, but did eliminate within path dependence by using the median correlation for studies which reported more than one correlation per path. No corrections for attenuation were used, nor were they'mentioned in the text. Schmidtl Hunter and Outerbridge. Schmidt et al. (1986) examined a path analysis of the impact of job experience on job knowledge, with additional paths for the effects of mental ability, work-sample performance and supervisory ratings of job performance. They had 4 studies of these 5 variables from military settings, and. corrected for attenuation in, all studies. Schmidt, Hunter and Outerbridge conducted a path analysis on this data, and found path coefficients along each path of their theoretical model. They averaged the disattenuated correlations across the 4 studies, then fit the path.model. They did.not conduct a test of overall model fit, nor did they consider the dependence in the data. For 3 of the five variables, the reliabilities were not based on sample values. For work samples and supervisory ratings, the reliabilities for all 4 studies were set to .77 and .60, respectively. These values were determined from weighted averages of reliabilities from several studies, not including the ones used in this synthesis. The reliability for job experience was assumed to be 1.00, as the records :Lndicated the number of months on the job. Th: reliabil research al., ig: and the The reli their und rather t? PreJ several 1 multivarj unioniZat On EVEry tested a every pa; One Pair with the: stUdies 15 This study illustrates the main problem with obtaining reliability information. Often, estimates based on past research or other samples within a study are used. Schmidt et al., ignored the uncertainty in the reliability coefficients and the dependence between correlations and reliabilities. The reliabilities in their study were sample estimates, so their uncertainty should have been considered in the analysis, rather than considering them to be fixed. Premack and Hunter. Premack and Hunter (1988) examined several univariate relationships and used them to create a multivariate (causal) model of employee decisions about unionization. In effect, they did a univariate meta-analysis on every path in the model, then combined the results and tested a causal model. However, not every study examined every pair of variables and some studies examined more than one pair. While they provided an overall test of model fit with their method, the dependence between relationships within studies was ignored. Premack and Hunter used univariate attenuation corrections for each individual correlation. Becker and Schram. Another simple case to which this methodology might apply considers the data from Friedman (1991). In this example, data from several studies were collected to assess the relationship between math, verbal, and spatial abilities. An existing model hypothesized relationships among all three variables. Several studies were collected which contained correlations for all three (I? (D (h) -_ multivar univaria separate meta-aha tEStS for whether 6 did th d consiGEret All < OEtEn 1'th exiSt_ 16 relationships. Mathematical ability was used as an outcome with the other two variables (verbal and spatial ability) used as predictors. Becker and Schram (1993) conducted both univariate and multivariate syntheses on a subset of this data. Their univariate analysis examined. the three relationships separately (between each pair of variables) using traditional meta-analysis procedures, including conducting homogeneity tests for each path. The results from that analysis indicated whether each path was homogeneous, and.if so, the magnitude of the average correlation. Although beneficial, these results did.not directly test the model posited. 'There was no overall test of fit, and the interrelationships among the paths (and dependence in the data) were ignored. The multivariate synthesis allowed the examination of the effects of each predictor on the outcome, and on each other simultaneously. Tests of significance and the relative importance of each predictor were also examined and prediction equations were formulated. This analysis modeled dependence in the data and gave a more complete assessment than the univariate analyses because partial relationships were considered. All of these examples point out methodological problems in doing this type of synthesis. The dependencies in data are often ignored, or unmodeledfl Other statistical problems also exist. Measurement errors have yet to be completely conside: Outerbr: did not reliabi; Schram ( four stt of synthj to be dc: synthesis 17 considered” Premack.and Hunter (1988) and.Schmidt, Hunter and Outerbridge (1986) corrected for attenuation on each path, but did not account for dependencies among the correlations or reliabilities. Harris and Rosenthal (1985) and Becker and Schram (1993) did not correct for measurement artifacts. All four studies attempt to address the problem in meta-analysis of synthesizing only main effects. However, much work needs to be done to solve the problem of the best way to do such a synthesis. Validity Generalization and Corrections Validity generalization (VG) is an approach to combining correlational study results which grew out of interest in the power of employment-selection measures to predict job success. VGV has focused. on correcting for so-called "artifactual variation" in studies, including measurement error. Schmidt and Hunter (1977) claimed that the variability among study results could be attributed solely to sampling error when measurement errors are eliminated. Their work has focused on the development of corrections for attenuation and range restriction in the synthesis of one outcome variable (bivariate relationships). However, reliability information is often unavailable in published studies. Thus, Hunter and Schmidt (1990) (and cathers) have sampled from hypothetical distributions of reliabilities in making corrections. Distributions given by Pearlman, Schmidt, and Hunter (1980) are often used when reliabi includi: mention 1h reliabi; validit} the by; 18 reliability information is missing in studies. Others, including Raju, Burke, Normand, and Langlois (1991) have mentioned potential problems with these hypothetical reliability distributions. In particular, the accuracy of validity generalization procedures is affected.by how closely the hypothetical distributions ‘match. the real population distributions (Paese & Switzer, 1988). Because the real distributions are not accessible, such a match is almost impossible to establish” .Also, these hypothetical distributions were derived for the literature on personnel selection, which may not represent the distributions of reliabilities found in educational or social-science data. Reinhardt and Mendoza (1989) also questioned the use of these hypothetical distributions. They claimed that the hypothetical distributions could be unrepresentative of the real data, and that there were no guidelines to assess the accuracy of the hypothetical distributions. As a result, they focused on using traditional VG procedures with "situational data" rather than with hypothetical data. They used reported reliabilities from samples in other studies when calculating unknown reliabilities. Their new procedures were fairly accurate when as much as 5095 of the reliability data was missing, suggesting that the procedures did not require a great number of studies. A few quality studies were sufficient to produce accurate reliability estimates. Ra~ reliabil develo 'U conside: reliabil reliabil (2.1). adjusted substitu‘ treated formulas 19 Raju et al. (1991) approached the hypothetical reliability distribution problem from a different angle. They developed a procedure for correcting correlations that considered the sampling error arising from approximating the reliabilities. Raju et al. (1991) used averages of available reliabilities as pxx and pW for making the correction in (2.1). The variance of the correlation coefficient was adjusted for the uncertainty that arises from making this substitution. When a reliability was reported, Raju et al. treated it as fixed in the derivation. of the variance formulas, even though it might have been based on a sample. When the reliability was from an average, it was treated as variable. The resulting variance in the correlation coefficient was larger when a hypothetical distribution was used, because of the additional variability in the reliabilityn Raju.et al. (1991) used simulation techniques to show that their method provided.more accurate estimates of the population correlation than other procedures. One other criticism of using hypothetical reliability distributions and simple averages to replace missing reliabilities is that, in both situations, reliability data are treated as missing at random. Hedges (1989) has argued that if low sample correlations are found in a study, the :researchers may be more likely to report artifact corrections 'than researchers who found high correlations. Also, studies vflnich focus on situations with greater economic and legal risk may be Reliabil at rand: Other U2! (EqUatio: 0f the tw Correlatj Kland Y 2 usual CI Correlat: In t nK‘ethOd h and the estimat '11 they fQU well, Dr T: .83; CCI‘; ‘ t \ arge‘sa: are USEd 20 may be more likely to monitor and report reliability. Reliability information therefore seems unlikely to be missing at random. Other Univariate Corrections HakstianLischroeder. and Rogers (1988). Hakstian et al. (1988) considered the variance and covariance of univariate correlations corrected using test-retest reliabilities differently than those who used the traditional method (Equation 1). They assumed that one would have two measures of the two variables, X1 and X2 , and Y1 and Y2, and the sample correlation would be the average of the correlations between X1 and Y2 and X2 and Y1 . After this average was computed, the usual correction was used to estimate the corrected correlation. In their study, Hakstian et al. derived, using the delta method, the variance:of this corrected (separate) correlation, and the covariance between two corrected correlations estimating the same phenomenon. using a simulation study, they found that the corrected correlations behaved fairly well, provided that the sample size was greater than 150. They concluded that correcting correlations seems to be a large-sample procedure, because three sample values with error are used to estimate the corrected value. While the results of this study relate to the present work, the situation examined is problematic. Reliability information is seldom reported in meta-analytic studies, and finding : be even ' present 5 consider technique correctio 21 finding data on two parallel measures of each variable would be even more unusual. Although the simulation used in the present study'considers test—retest reliabilities, it will not consider Hakstian et al.’s formulation because of its impracticality. Meta-analysis and corrections. More general synthesis techniques (meta-analyses) have also considered the use of corrections for attenuation and range restriction. Rosenthal (1984) recommended reporting both corrected and uncorrected results. He suggested that the majority of social-science researchers do not correct for measurement errors or report reliabilities, so uncorrected results are more typical. Hedges and.Olkin (1985) gave the basic correction formulas for both. mean difference effect sizes and for correlations, discussed the effect of making the correction on the variance of the correlation coefficient, and noted that their univariate methods apply to corrected or uncorrected correlations. However, Hedges and Olkin considered reliability values to be fixed and known, and therefore they did.not take the variability of the reliabilities into account when adjusting the variances of corrected correlations. This assumption seems unwarranted, given that as noted above, reliabilities are often missing and estimated, and may involve much uncertainty. making attenua' variable theoret; (? '1 DJ ’4. ('1 true ValJ such as reliabil: pOor Est. In PreSent pOPUlati| as imp0: v I p‘actlce h he‘Jer Cl! 22 Measurement Problems in Univariate Syntheses Winne and Belfry (1982) discussed several issues related to correcting for attenuation, including the reasons for making such corrections. 'The result of the correction for attenuation is an estimate of the true correlation between the variables of interest. This correlation represents a theoretical value, or, according to Winne and.Belfry, a latent trait” ‘Winne and.Belfry urged caution in interpreting results fromianalyseS‘which.use corrected correlations, and they cited several measurement specialists who share their concern (Allen & Yen, 1979; Cronbach, 1971). The concerns stem from the claim that the resulting corrected correlation represents the true value between constructs measured without error. Factors such as sampling error of observed correlations and reliabilities and correlated measurement errors may result in poor estimates of this true correlation. In the meta—analysis application resulting from the present work, the interest is in estimating a theoretical population value, so correcting appears appropriate. However, as important as the concerns expressed above seem, in practice, the true values are always estimated, and it is; never clear how much error is affecting such estimates. Another problem noted by Winne and Belfry (and others) was corrected correlations larger than unity. This problem was attributed to correlated measurement errors, the type of reliability, and the accuracy of the estimate of the -~-I reliaDL. m.g.,k and l) ( assumed correcte 'i‘AC model, c derivati Correc:g the dis: COEE f i Cf in the r81iabi: 23 reliability used (Winne & Belfry, 1982). Corrected correlations larger than. unity' were not acceptable, and adjusting for such potential problems in the multivariate case (e.g., by forcing the corrected correlations to be between -1 and 1) was necessary. When reliabilities are sampled from assumed distributions, the potential for out-of—range corrected correlations may be even greater. Thomas (1989) derived, based on a classical test theory model, distributions of corrected correlations. Using his derivation, Thomas addressed the issue of out-of—range corrected correlations. He suggested a procedure which uses the distribution function (and its inverse) of a correlation coefficient so that the corrected correlation is forced to be in the interval -1 to 1. His derivations assumed that reliabilities were known and fixed, and he claimed that viewing reliabilities as random variables would complicate the picture, and would not be likely to yield practical increments of improvement better than those he derived. Thomas claimed that if the sample size is sufficiently large, the difference between estimates based on fixed versus random reliabilities should be negligible. He also stated that more work in this area was necessary, and the work here should answer some of the questions he raised. Also, this work examines the case in ‘which reliabilities are viewed as random. multiva* study 0“ corrects arises f based, : 24 Measurement Problems in a Multivariate Synthesis Multivariate syntheses will have their own set of measurement and statistical problems. Ihi univariate syntheses, covariances among variables are irrelevant. In a multivariate synthesis, covariances (i.e., between multiple study outcomes) also need to be corrected, or calculated with corrected correlations. Also, when more than one correlation arises from(the same sample, and the reliabilities are sample- based, the potential for correlated measurement errors exists. When the reliabilities and correlations of interest are calculated for the same sample, the observed reliability and the correlation are dependent. This dependence becomes mOre problematic when the situation is multivariate, because the reliabilities for different tests could also be interrelated when they are determined from the same sample. .Accounting for covariances between (1) reliabilities for two different measures from the same sample, and (2) between a reliability estimate and the correlation it is used to correct, is a further problem that is considered in the present work. Estimating correlated measurement errors within a set of data is another issue which needs to be addressed. The effects of such errors should be considered. Although not exactly an attenuation issue, this is a measurement-error issue with practical applications. Extensions of the work of this dissertation could consider such issues. Mt studied Fuller similar considei Also, ch are oft techniqx and not E; derived the rm applied estimat reliab: predict of bot] 25 Existing Multivariate Corrections Multivariate corrections for attenuation have been studied in the regression literature (e.g., Gleser, 1992; Fuller & Hidiroglou, 1987). While these corrections are similar to ones that may apply to meta—analyses, they do not consider many of the practical problems of meta-analytic data. Also, they make assumptions about the nature of the data which are often violated in meta-analyses. For example, both techniques assume that the population reliabilities are known, and not based on sample data. Fuller and Hidiroglou. Fuller and Hidiroglou (1978) derived regression estimators of slopes based on correcting the raw moment matrix for attenuation. Their derivations applied to situations where the error variances are not estimated from the same data ‘used to estimate the reliabilities. They assumed that reliabilities for both the predictors and the criterion are known. They addressed cases of both correlated and uncorrelated errors. Fuller and Hidiroglou’s method corrected the moment matrix using a diagonal reliability matrix. This matrix used (1 - reliability) as the basis for the correction. The quantity (1 - reliability) is the ratio of error variance to the total variance in the predictor. By pre- and post- multiplying this reliability matrix by a diagonal matrix containing the standard deviations of the predictors, the variances were adjusted for the errors in measurement. This COIIECZ' regress Their p: is posiq Alp in multi aSSESS 26 corrected moment matrix was then used in the traditional regression calculations for estimating the regression slopes. The authors then examined the distributional properties of this corrected matrix, and of the corrected estimators. Their procedure guarantees that this corrected moment matrix is positive definite. Gleser. Gleser (1992) examined measurement reliability in multivariate regression. He stated that if the goal is to assess the relationship among the true (latent) variables, then classical least squares estimation yields biased and inconsistent results. IHis errors in variables regression (EIVR) procedures provided alternative methods of estimation. Gleser's approach used prior information about both the reliabilities and the data to estimate a reliability matrix, which is then used in estimating the regression slopes. Unlike Fuller and Hidiroglou’s approach, in Gleser's method his reliability matrix contained more than the reliabilities of the predictors. The reliability matrix also contained the correlations among the components of the reliability values (true and error variance). Gleser did not consider ‘measurement error in the outcome variable or variables, and he assumed that the outcome and the measurement errors in the 'predictors were uncorrelated. In. Gleser’s ‘method. the estimate of his reliability xnatrix, A, comes from previous information about the Ixredictors, generally taken from other reliability studies. Gleser’ predict. reliabi. tradit': general: Psychome Eigenvec error on Boc \ abOVe di Bock and used max, TESultinS ThEy base and On formulatj 1975); he bEQn Stllc BOG}: Hidiroglc gaar d antes m, . Variance ‘ 27 Gleser’s preferred reliability study would.consider all of the predictors to be used in the regression model. ~His reliability matrix, A, iS' then used in estimating the traditional regression model by multiplying the predictor matrix by the inverse of A in the estimate of the slope. If least squares estimation is used, this result is a generalization of the correction for attenuation used by psychometricians. Gleser uses the eigenvalues and eigenvectors of A to assess the influence of the measurement error on the accuracy of the estimates. Bock and Petersen. The regression formulations described above differ somewhat from that of Bock and Petersen (1975). Bock and Petersen’s multivariate correction for attenuation used maximum likelihood estimation to make certain that the resulting variance-covariance matrix is positive semidefinite . They based their formulation on classical-test—theory models, and. on having' a known. measurement-error matrixg Their formulation has been applied in studies (see, e.g., Petersen, 1976); however, the effects of using their correction have not been studied. Bock and Peterson’s method is similar to Fuller and Hidiroglou's, since both rely on adjusting eigenvalues to guarantee positive definite matrices. Bock and Petersen's derivation was based on true- and error-component covariance 'matrices. A.restricted maximum likelihood estimate of the ‘variance-covariance matrix was the result. Bock and Petersen did not values. the obs matrix ( at leas where t3 the tra: Correct: SYnthesi SituatiC COeffici COrreCt predictc 33mm w Ncn to add} multiVa 28 did not make any assumptions about reliability types or values. The roots of eigenvectors of the difference between the observed and error matrices (the solution to the two matrix eigen-problem) were used to ensure that the matrix is at least positive semidefinite. In the two-variable case where the measurement errors are uncorrelated, the result was the traditional Spearman correction. These methods were all slightly different, and may be applicable to different situations in the synthesis of correlation matrices. However, it is not clear if the corrections used in regression analyses can be applied to synthesis situations. For example, corrections in regression situations were applied to raw data (not to the correlation coefficients). Second, some of the corrections did not correct for measurement error in.the relationships between.the predictors and the outcome. Summary None of the research mentioned previously has attempted to address the role of corrections in the synthesis of multivariate correlational data, and, therefore, take into account the problems mentioned previously. No meta-analytic studies address the issue of correlations among errors for different variables within the same study. This study focuses on methods for correcting correlation matrices from individual studies and computing the associated variance-covariance matrix of the correlations for each study. Once ea result: multiva details , I the issd To und» correla: VECCOI‘S 29 Once each study’s correlation matrix is corrected, and its resulting variance-covariance matrix is found, then a multivariate synthesis can be completed. The next chapter details the statistical notation and theory needed to analyze the issues involving multivariate corrections for attenuation. To understand the statistical problems with correcting correlation matrices, we must examine the distributions of vectors of corrected correlation coefficients. multiva: studies denoted Correlat sample a CHAPTER III METHODOLOGY Statistical Formulations Notation Let £1, ...., 3p be random variables with the multivariate normal distribution, and let the number' of studies which examine correlations among these p_variables be denoted as 5. There are p;*_=p(p-1)/2 non—redundant correlations possible in any study. Let Eist and pist be the sample and population correlations between 58 and gt for the ith study, where g and t = 1 to 21, and i = 1 to 3. Let p’ist represent the corrected sample correlation defined by (2.1) and assume that each study contains only one measure of each construct or variable of interests The sample reliability for a measure of variable s will be denoted.;ass. The number of people in study _i_ will be denoted ni. In matrix notation, let :1 represent the vector of observed correlations (r112, rim,..,rilp,...,ri(p_1)'p) and let p'i represent the vector of corrected correlations, and p1 the vector of population correlations. Let V(ri) be the variance-covariance matrix of the observed correlations, V(p’1) the corrected variance-covariance matrix, and V(p1) the population variance- covariance matrix for the correlations. The reliability matrix or any matrix containing corrections based on reliabilities will be represented using A1. 30 notati algebra statist correlai 31 Multivariate Analysis Multivariate analysis generally involves multiple outcome (dependent) variables and several predictors. bhnfln of the notation and analysis comes from matrix algebra, and several algebraic properties of matrices are important in the statistical analyses. For example, variance—covariance and correlation matrices are known to be positive definite. This means that they are invertible and their determinants are nonzero. Knowing that these matrices must be positive definite will help determine whether the corrections attempted in this work are giving appropriate results. The existing literature on multivariate regression corrections is concerned with correcting matrices containing raw data or slopes, rather than correlation matrices, the focus of this work. The resulting changes in relevant variance-covariance matrices from using these other methods (e.g., changes in slopes) are different from the traditional corrections for attenuation. One way to assess the consequences of the correction for attenuation is to examine the determinant of the variance- covariance matrix for the corrected correlations and that of the corrected correlation matrix itself. The determinant is a function of the elements of a matrix, and the determinant shows whether a matrix is invertible (and positive definite). Estimates of variance-covariance and correlation matrices are invalid if they are not positive definite. (1‘ Distr; ls l correla where rl correlai correctJ predict! conside: variance 2 i8 gix Bo} the dis ThEy f reliabi ThEir 32 Distributions of Corrected and Uncorrected Correlations Variances. The large-sample asymptotic variance of a correlation coefficient for a sample of size Qi is given by Var(r_ist) = (1 - 915:2)2/111 , (3.1) where rast is the sample correlation, 249s is the population correlation, and Qi is the sample size. The variance of the corrected correlation (corrected for unreliability of both the predictor and the criterion) differs from this because it considers the covariances between the reliabilities and the variances of the reliabilities. 'The variance of the corrected ; is given in Appendix A. Bobko and Rieck (1980), among others, have investigated the distributions of functions of correlation coefficients. They found that correlations corrected. using known reliabilities are more variable than uncorrected correlations. Their results ShOW’ that simply' substituting' a corrected correlation into the formula in (3.1) to compute the variance of corrected correlations can give misleading results. Covariances. No research was found which showed investigations of the behavior of corrected variance- covariance matrices of correlations. The variances and covariances of univariate corrected correlations can be derived using the delta method (Rao, 1973). Although these derivations take into account the covariation between correlations, they still do not consider whether the resulting varianc definit Th do not follows Cov 33 variance-covariance matrix of the correlations is positive definite. The covariance between two correlation coefficients which do not share a common index (the most complicated case) is as follows (Olkin & Siotani, 1976): COV (gist I l:iuv ) = [0'5 pist piuv (pisu + pisv + pitu + 2 pitv ) + pisu pitv + pisv pitu ' (pist pisu pisv + pits pitu pm + pius pm pm + pm pm pm )l/si- (15-2) This equation simplifies when the pair of correlations share an index; Appendix: B contains the covariances between correlations for different cases, including covariances between reliabilities, and between a reliability and a correlation, based on the formulas found in Olkin and Siotani (1976). Appendix .A shows the variance of a corrected correlation and the covariance of a pair of corrected correlations which were derived using the delta method for the simplest case, a 3 x 3 correlation matrix. These corrections lead to new variances and covariances. The corrected variance-covariance matrix is obtained by pre- and post- multiplying the uncorrected variance-covariance matrix by the matrix of first derivatives (the Jacobian) of the functions of the corrected correlations used in (2.1), which consider both the sample correlation and the reliabilities as random variables. The result of this multiplication is a matrix which c correla Data Gel A problem Of van COII‘ECt. generate Simulat; leed, I and err. examine: I‘ESulti] COVarial 34 which contains the variance-covariance matrix:of the corrected correlations. Layout of Simulation Data Generation A simulation was conducted to examine the frequency of problems (such as overcorrection and non-positive determinants of variance-covariance matrices) occurring because of corrections. This simulation used multivariate normal data generated with uncorrelated measurement errors. In the simulation, population correlations and reliabilities were fixed, then data were generated for each distribution of true and error scores. Observed and corrected correlations were examined, as well as sample reliabilities and determinants of resulting corrected correlation matrices and variance- covariance matrices among the corrected correlations. This simulation examined the simplest posSible multivariate case, based on three population correlations (p12, p13, p23) which arise from three variables (51, £2, and 53) , and the resulting three sample correlations (r12, £13, and £23) . The corresponding sample reliabilities are £11, £22, and £33- Two different methods for simulating the results were used. One, based on classical test theory, begins with true score and.error variances, and then.computes reliabilities and true correlations based on these values. A simplifying assump: equal i (1991). the thr third (\ first t correla 3S assumption of letting the variance of the observed scores equal 1 was used. This method was also used by Raju et al. (1991). When applying this method to multivariate data, in the three variable case, two correlations were fixed, and the third correlation was based on the relationship between the first two correlations. 'The results indicated that this third correlation varied around a fixed value (as would be expected). In order for the correlation matrix to be positive definite, there is a distinct relationship between the three related correlations. The interval of possible values for the third correlation (rg3), given the first two (:42 and :13), is centered around the product of the first two correlations: £12*£13 +/- x/((1-;212)*(1-;213)) (Stanley & Wang, 1969) . The other method of simulation used the multivariate normal generator' in. IMSL (International. Mathematical and Statistical Library). A desired variance-covariance matrix for the observed scores was derived from the known true and error variances, based on the population reliabilities, and under the assumption of unit variances for the observed variables. The variance-covariance matrix for the true scores is P11 (012 Vpii V922 913 V911 "4033 912 vpll V922 922 923 «(’22 V933 913 V911 V93 923 V922 V933 P33 — The va 36 The variance-covariance matrix for the errors is — — v With this method, the matrices of the three population correlations and. the three jpopulation. reliabilities were specified ahead of time. The multivariate normal generator then provided data with the given variance-covariance structure, from. the Cholesky factorization. of these two matrices. Once the data are generated, the true and error scores are summed, to yield observed data with the desired properties. Both methods gave similar results in basic simulations. The method using the multivariate normal generator was chosen for further use, because it allowed the specification of the third correlation” The third correlation for the data used in the simulation was always in the range of values specified above. Different Univariate Corrections Differences in simulation results are found when considering which reliability definition or correction formulation to use. Raju et al. (1991) showed the derivation of corrected variances for the univariate case, in which they defined the corrected correlation to be C _ r W — rxy , (3 . 3) rxxt ryyt where popula result respec differ provid- formul. simulai scores are ne\ reliab; Tl 37 where rut = V320: and ryyt = \/_r_ yy (the square root of the usual population reliabilities). This formulation led to different results, because Raju et al. took derivatives of r£xy with respect to Ikxt instead of Ekx’ The variances they obtained differ from those in Appendix A. The fermulation in (3.3) provides much easier derivatives, but it is unclear which formulation is really more practical or accurate. For a simulation, the correlation between true scores and observed scores is readily available, but in practice, the true scores are never known. The correlations are only estimated, as are reliabilities. (The distributions of these two reliabilities (Ikxt and ggyt) and the effects of these two different formulations were examined. As expected, the distributions of rut and _ryyt were much more negatively skewed than the distributions of the Ekx and.;yy. Preliminary simulations showed that the formulation from Raju et a1. gave variances of corrected correlations that were far smaller than those expected based on the sampling distribution. Also, one can show that in Raju et al.’s formulation, V(_1:°) would always provide smaller variances than those corrections shown in Appendix A (V(p')). Therefore, further use of this formulation was not warranted, and it was not included in the final simulation study. Simulation Parameters Sample sizes. The size of the sample will have an influence on the magnitudes of the variances and covariances of the this s simula proble 1994), relati 100, 2 V3 study one of PIEdici sO tha‘ Correlg Could L the Cc combina repress and th. reprESe 38 of the correlation matrix. Because the derivations used in this study are based on large-sample theory, and because other simulation results have shown that small sample sizes present problems with covariance-matrix estimation (Becker & Fahrbach, 1994), the sample sizes chosen for the simulation are relatively large. The sizes chosen for this study are 50, 100, 250, and 500. Correlations. The correlation triples chosen for this study are based on practical regression situations in which one of the variables is an outcome, and the other 2 are predictors. The correlation triple (.00, .00, .00) was used so that the simplest case was represented. The rest of the correlation triples represent various population outcomes that could underlie data in regression studies. Table 1 displays the combinations used, along with the R2 value for each combination. The first two correlations in the triples represent the population correlations between each predictor and the outcome. The third correlation in the triples represents the intercorrelation between the two predictors. The triples show varying degrees of relationship with the outcome, from weak to strong, and varying degrees of intercorrelation” 'The table shows that the percent of variance explained, using the second and third variables to predict the first variable ranged from .00 to .77. Many of the R2 'values are moderate, as would be expected in 39 educational situations. All triples are possible, given the constraints mentioned previously. Table 1 Simulation Parameters Sample Sizes: 50, 100, 250, 500 Correlation Triples: (o, o, 0) R2 = .00 (.4, .3, .1) R2 = .23 ( 4, 3, 7) R2 = 16 (6, 4, - 2) R2 = 64 (.6, .4, .2) R2 = .44 (7, 6, 1) R2 = 77 ( 7, 6, 8) R2 = 49 Reliability Triples: (.7, .7, .7) (.85, .85, .85) (.9, .8, .7) (1, l, 1) on pr provi other lowes measuj admin. reliai when c .70 (1979: were 1 distri Reliab Superv 4O Reliabilities. The reliability triples also were based on previous research. First, the triple (1.00, 1.00, 1.00) provides the case of measurement without error. The three other triples show varying degrees of unreliability, with the lowest triple (.70, .70, .70) representing moderate measurement error. 11 search. through several nationally administered standardized tests showed that subtest reliabilities typically ranged from .85 to .95. Therefore, when considering standardized achievement type measures, ( . 70, .70, .70) is lowu IHowever, Schmidt, Hunter, Pearlman, & Shane (1979) provided hypothetical criterion reliabilities which were much lower. These seemed to have a roughly normal distribution centered on .60 and ranging from .30 to .90. Reliabilities this low may be representative of employment criterion reliabilities, when the outcomes are often supervisory ratings. In education, reliabilities typically do not appear that small. In fact, hypothetical reliabilities for predictors given in Schmidt et al. (1979) ranged from .50 to .90, with 90% of them equal or above .75. This range of reliabilities seems more consistent with the educational literature. Examining reliabilities from test manuals seems more reasonable than arbitrarily using hypothetical distributions found in employment literature. For example, Bock & Vandenberg (1968) used test manuals from the Differential Aptitude Test to give the error variances (reliabilities) used in their study which used a multivariate COIIE COIIE poss: OVGIC OCCUI with ; 41 correction for attenuaticmL All sample sizes, reliability and correlation triples were completely crossed so that each possibility was considered. It is expected that some overcorrection (corrected correlations greater than 1) will occur when the (.70, .70, .70) reliability triple is paired with population correlations greater than .70. The number of replications used was 2000. Given the parameters of the simulation, there are 4 X 7 X 4 = 112 cases to be considered. Four methods (Fuller and Hidiroglou, Bock and Petersen, Gleser, and univariate) of correcting correlations were considered, and their variance-covariance matrices were computed. The limitations of the variance- covariance matrices are discussed below. The sampling distribution for each case was also examined. Basis for Comparing Methods of Corrections Corrected correlations. From the multivariate corrections given by Fuller and Hidiroglou (1978) and.Bock and Petersen (1975), it was possible to compute a corrected correlation matrix. Appendix C contains descriptions of the various methods for finding corrected correlations. The corrected variance-covariance matrix of the raw scores was computed, and then converted to a correlation matrix. Comparing to the univariate corrected correlations in the simulation was then straightforward. The average magnitudes of the differences between the population values and these corrected correlations showed the difference between methods. C) In matr the multi Hidi: Vari; Were CorJ the the p03: Car: the 42 In Gleser’s method, while an estimate was determined for the matrix lambda (of errors/reliabilities), it was not clear how the corrected variance-covariance among the raw scores was adjusted. However, corrected correlations were found. All of these correction methods were computed in the simulation. Because the simulation starts with raw'data, all of the needed corrections were then obtained. Variance-covariance matrices. None of the three multivariate corrections (Bock & Petersen, 1975; Fuller & Hidiroglou, 1978; Gleser, 1992) provided estimates of a variance-covariance matrix for the corrected correlations. However, the method from Fuller and Hidiroglou (1978) did allow'for a derivation of a variance-covariance matrix for the corrected correlations. This computation was a variation of the large—sample theory variances and covariances found in Appendix A. Forms of the other two methods (Bock & Petersen; Gleser) were not amenable to such a calculation. The Gleser correction.was very similar to the univariate correction, and the variance-covariance matrix for this case was identical to the large-sample theory method, therefore no computation of variance for this correction was attempted. The correction from Bock and Petersen.did.not lead to any possible correction to the variance-covariance matrix of the corrected correlations. The only possibility was to insert the corrected correlation from Bock and Petersen into equatior matrix cP results, The of the t: and cova; in the Finally, This met? resulting reliabili V( \ Of the CC The their Va] methods UniVaria‘ 43 equations (3.1) and (3.2) to estimate the variance-covariance matrix of the corrected correlations. This procedure yielded results similar to the univariate method variance calculation. The fourth and fifth methods to be compared were the use of the univariate corrected correlation in the given variance and covariance formulas (Equations 3.1 and 3.2), and.their use in the large-sample theory formulas shown in Appendix A. Finally, treating reliabilities as constant was also examined. This method involved using Equation (3.1) and multiplying the resulting variance by the inverse of the product of the reliability values, i.e., V(p'xy) = .1 * V(rx ). pxxpxy The covariances were calculated similarly. The comparison of these six ways of calculating the variance-covariance matrices of the corrected correlations is shown in the next chapter. Comparisons Made W The exact comparison of formulas for the corrections and their variance-covariance matrices for the different methods was difficult. The first examination of these methods determined how they compared for the simplest case: 2 variables, 2 reliabilities, and 1 correlation. All three methods were compared to one another and to the typical univariate correction (Equation 2.1). Bock and Petersen's (1975) q correct: equivale and Hidi NGX‘ the thre exact CC equivale Variance §l§ulggg Th. percent analid COVaria matrix to One Simulat distrik Show hc Closes: Met. a ~i\::L: 2: Of met This C: but a1 44 (1975) correction claims to be identical to the univariate correction for this case. Claims are not made about the equivalence of the other two corrections (Gleser, 1992; Fuller and Hidiroglou, 1978). Next, the three variable case was examined. Differences from the above situations were expected, and the results from the three methods were not comparable. The result of these exact comparisons showed whether any of the methods provided equivalent, larger, or smaller corrected correlations and variance-covariance matrices. Simulation Comparisons The first results of the simulation study display the percent.of out-of-range corrected correlations, the percent of invalid (non-positive definite) correlation or variance- covariance matrices, and the corrected variance-covariance matrix for each method and case. These results are compared to one another and.to the sampling distribution created.by the simulation. The differences between the empirical sampling distributions and the observed values from1each.of the methods show how the methods differ and which method(s) give results closest to the empirical sampling distribution. WW After the simulation results were completed, a comparison of methods was made using data from a previous synthesis. 'This comparison not only examined the corrected correlations, but also showed whether the conclusions of the data analyses changed, reconsic original 45 changed, depending on the correction used. The analysis to be reconsidered is from Becker and Cho (1994) , though the original data are from Schmidt, Hunter and Outerbridge (1986) . Exa Gleser Hidirogl algebrai 2 relial and PEte Correctj correct: 1 \ standerc undEr th IEpreSer COVaria: varianCe predicto CHAPTER IV RESULTS Exact Comparisons among the Methods Exact work was used to compare the methods (univariate, Gleser (1992) , Bock and Petersen (1975) , and Fuller and Hidiroglou (1978)). The four' corrections ‘were compared algebraically. First, the 2 variable case, with 2 variables, 2 reliabilities and 1 sample correlation was examined. Bock and Petersen’s (1975) claim that for this simplest case; their correction was the same as the traditional univariate correction, was verified. Gleser (1992) The correction from Gleser (1992) also always gives the univariate correction in the cases which were considered here. The reliability matrix (A) in Gleser is an adjustment to the usual sum-of—squares and cross products (SSCP) matrix. The SSCP matrix is p multiplied by the variance-covariance matrix of the raw scores if the sample mean is 0 and the sample standard deviation is L. The simulation.studied here operated under those assumptions. Gleser (1992) lets A = 2'10,” * 2“.“ represent the reliability matrix, where E'lob. is the variance- covariance matrix of the observed predictors and Etna. is the variance-covariance matrix of the true scores for the predictors. Then, the adjustment in the regression case is 46 (3:41 when mul for the mean of C look like true - where (x 47 fl = A71 * (X'X)'1 X’Y (Gleser, 1992). The adjustment occurs when multiplying the inverse of A by (x'x)'1. Elementwise, for the three variable case, with two predictors (assuming a mean of 0 and a variance of 1 for all variables), the matrices look like: 1 "r21 -1 2 2 Eobs = 1 " r 23 1 " r 23 and 'r23 .4 Etrue = P22 r23 r23 P33 where (X'X) = n r53 n ‘ r53 n n The adjustment is occurring only to the predictor variables. The variance of the true scores is equal to the reliability when the variance of the observed scores is assumed to equal 1. The corrected correlation is then found by dividing the off-diagonal element of (X’X)n" by the product of the square roots of its diagonal elements where (x'x)new = (4'1 * (x'xr1r1. where A} gives cc correla: Bock and The univaria well def correlat further Correcte Observed COIIECtic Boc) matriceS of Squar. are the CI‘OSS pr their re SOlving “S 48 where A'1 = (E'lob. * Bum)’1 The adjustment using A then gives corrected.correlations identical.t0>the usual univariate correlations. Bock and Petersen (1975) The approach of Bock and Petersen (1975) gives the usual univariate correction if the original correlation matrix is well defined (non-negative definite), but if the corrected correlations are greater than unity, the matrix is modified further so it becomes positive definite, and a different corrected correlation is produced. This method increases the observed correlations, but not in the same way as the usual correction for attenuation. Book and Petersen’s method manipulates the moment matrices, M; and M&, the mean error and the mean observed sum of squares and cross products matrices, respectively. These are the matrices found by dividing the sums of squares and cross products matrices for the error and observed scores by their respective degrees of freedom. Their method involves solving the two matrix eigenproblem (M? - Xi M.) xi = 0 (Bock and Petersen, 1975, p. 674). Once this problem is solved, the estimate of the true variance-covariance matrix of the raw scores can.be made using the following formulation. Let X = (xi, ... xp) be the matrix of eigenvectors, let A' = diag(ki, ..., AP) be the matrix of eigenvalues, and let Ip be the p x p identity matrix, then * Et=MY-M.=B’ (A -Ip)BwhereB-—=X'1. If any 6- are ref; be at leg then cal Fuller a Ful and Pete adjust t; that the be slight and Hidi correlatj Fuller ar fI‘Om the matrix WC The uanariat how it Co Where H-l variables of Stand: 49 If any of the elements of the A.matrix are less than one, they are replaced by 1.0 in the calculation, This constrains St to be at least positive semidefiniteu Corrected correlations are then calculated using the elements of 2t. Fuller and Hidiroglou (1978) Fuller and Hidiroglou’s (1978) method is similar to Bock and Petersen’s (1975) in that it also uses eigenvalues to adjust the corrected correlation matrix“ However, it appears that the corrected correlation from Fuller and Hidiroglou will be slightly different from that of Bock and Petersen. Fuller and Hidiroglou’s method considers the already-corrected correlation.matrix.and forces it to be positive definite» The Fuller and Hidiroglou method only gives corrections different from the univariate correction.when the corrected correlation matrix would be non-positive definite. The Fuller and Hidiroglou method is the same as the univariate correction in the 2 x 2 case. The following shows how it compares. 'The Fuller and.Hidiroglou method starts with the regression equation )3 = 3'1 (n'1 x' ‘1), where H‘1== Ufa X’X) - D A D, and where p and x are two variables in the x matrix and where D is a diagonal matrix of standard deviations of p and x, Also, A values: Then, f: assumed Simplifi( for the 801‘!ng If definits produCt 50 Also, A is a diagonal matrix containing 1 - reliability values: 1 — rhw 0 A = 0 1 - rxx Then, for the case where the mean of each variable is assumed equal to 0 and the standard deviation is 1.0, H simplifies to: 32 s 32 (1 r o W wa This in turn yields the new variance-covariance matrix for the raw scores 2 erxx Solving for the correlation between 5 and p gives the usual correction for attenuation rXW thx‘erw If the matrix of corrected correlations is not positive definite, the adjustment comes from pre-multiplying the DAD product by the quantity (f - rfl), where f is the smallest This prc Fuller a: definite The case pro: the univC were impt Correcti: HOWever' Mlnitab . Simuldti Fir VEry Sim CaSes. correCtj large 86 and Ful they 8i COrrela correCt 51 root (eigenvalue) in the two-matrix eigenproblem | M - fCGC ) = 0. Here C is the matrix of standard deviations of the raw scores and G a diagonal matrix containing reliability values. This procedure is similar to Bock and Petersen’s. However, Fuller and Hidiroglou’s estimate is constrained to be positive definite, while Bock and Petersen’s is positive semidefinite. The Three Variable Case The formulas for the corrections in the three variable case proved to be excessively complex for all methods (except the univariate case). As such, algebraic (exact) comparisons were impossible to make. In other words, comparisons of the correction formulas did not lead to any clear conclusions. However, several small-scale examples using spreadsheets and Minitab were computed, and the following results (before the simulation part of the study was conducted) were noted. First, the Gleser method yields corrected correlations very similar to the traditional univariate correction, in all cases. The difference between the Gleser and univariate corrections is minimal, with the difference near zero when large sample sizes are used. Second, Bock and Petersen (1975) and Fuller and Hidiroglou (1978) have similar methods, but they give different corrections; when the usual corrected correlation matrix is not positive definite or contains corrected correlations greater than.1ufiran The simulation results show which method gives larger corrected correlations, and whit the sim‘ Th COI'I'ECCE: and kno' the uncc reliabil that th Variance (3.1) a the rel §EEE§§y Th ShOWs (thOSe 52 and which variances are closest to the expected variances in the simulation. The results of the computation of the variance of the corrected correlations, assuming that reliabilities are fixed and known values are also of interest. In this case, the variance of a corrected correlation is simply the variance of the uncorrected correlation divided by the product of the reliability values. This variance of a correlation (assuming that the reliability is fixed) should be larger than the variance of a univariate corrected correlation computed.using (3.1) and (3.2), unless the reliability values are 1.0. If the reliabilities are 1.0, the two variances will be equal. Summapy of Exact Results This examination of the exact results from each method shows that, for legitimate corrected correlation matrices (those that are positive definite), all 4 methods produce the page values for the corrected correlations. If, however, the corrected correlation matrix would be non-positive definite, the Fuller and Hidiroglou and Bock and Petersen methods further adjust the correction. The simulation results should reflect this exact work, and additionally will show the magnitudes of differences among the corrected correlations, and the corresponding differences in the variance-covariance matrices of the corrected correlations. Bef the dif violated analytic three mu Ful situatio | the same that IEl are know 53 Assumptions of Each Method Before the simulations were conducted the assumptions of the different methods were examined. The methods often violated assumptions that would need to be made in meta- analytic studies. The following details were noted, and the three multivariate methods were compared and contrasted. Fuller' and. Hidiroglou's (1978) derivations apply' to situations where the error variances are not estimated from the same data used to estimate the correlations. They assume that reliabilities for both the predictors and the criterion are known. This assumption was violated in the simulation since the data used to estimate the reliabilities were also the same as those used to estimate the correlations, however, the dependence was accounted for in the calculation of the variances and covariances. Gleser's (1992) approach uses prior information about reliability values to estimate a reliability matrix which is then used in estimating the regression slopes. This reliability'matrix:contains more than.the reliabilities of the predictors. Ittalso takes into account the correlations among the components of the measurement error and also the correlations among the components of the true vector of predictors. Both Fuller and Hidiroglou's (1978) and Gleser’s (1992) corrections occur in regression models, and both make assumptions that would not necessarily be reasonable in meta-aha analysis the sampl these mi correlat) nonrando: Boc matrix, multivar maximum resultin is not reliabi] they us) human CE (1968) varianC Sample PetEISE AZ of TUI: Part 0: into t3 54 meta-analysis. In most primary studies included in a meta- analysis, reliability information (if given at all) is from the sample, thus is not a population parameter. .Also, both of these methods correct a raw data matrix rather than a correlation matrix, and both assume reliabilities are nonrandom. Bock.and.Petersen (1975) considered.the‘whole correlation matrix, rather than the individual elements. Their multivariate correction for attenuation uses restricted maximum likelihood estimation to make certain that the resulting variance-covariance matrix is positive definite. It is not clear what assumptions they make about the reliabilities. However, in the example given in their paper, they use a known value for estimating measurement error of human characteristics. In another study, Book and Vandenberg (1968) have used known reliabilities in estimating error variances. It is unclear whether using reliabilities based on sample data would violate any assumptions for the Bock and Petersen procedure. Although the methods discussed violate some assumptions of multivariate meta-analyses, all were used in the simulation part of this studyu Putting all of the corrected correlations into the derived formulas for the variances and.covariances of the correlation matrices produced variances and covariances which were compared to the sampling distribution. The effect of violating these assumptions (if any) was then determined. Caveats The version z summariz- during t above, t} the trac Program, sample C (2.1). f data mat: differenfi Sect for the \ eXiSted. COVarianC W8re USEC VarianCe- variaHCe‘ adjusted alSo with fiXed re: compared ( distribut 1n the Si 55 Simulation Results Caveats The simulation program was written in FORTRAN, and a version of this program is given in Appendix D. 'The data were summarized using a SAS program. Some assumptions were made during the data generation and analysis. First, as noted above, the:G1eser (1992) correction was virtually identical to the traditional univariate correctitmn However, in. the program, the univariate correction was calculated using the sample correlation and reliability statistics in Equation (2.1). The Gleser correction was estimated based on the raw data matrices, as shown in Chapter 3. Therefore, slightly different results were expected for these two methods. Second, although there were five different calculations for the variance-covariance matrices, only 3 unique methods existed. These methods are: (1) the traditional variance- covariance corrections using Equations (3.1) and (3.2), which were used to get the univariate and the Book and Petersen variance-covariance matrices, (2) the large-sample theory variance-covariance matrices (shown in Appendix A), which were adjusted for use with the Fuller and Hidiroglou method, and also with a traditional univariate correction, and (3) the fixed reliability calculation. These methods were then compared to the empirical variances computed from the sampling distribution of the corrected correlations across replications in the simulation. In correla: correcte variance encounte variance were sn proporti' correlat. variance were oft. when n = . 56 In each of the 112 simulation runs, counts of corrected correlations greater than unity, non—positive definite corrected correlation. matrices and. non-positive definite variance-covariance matrices were recorded. One problem encountered in this process was that the determinants of the variance-covariance matrices of the corrected correlations were small. Because the order of these matrices is proportional to 1/p3, and because variances and covariances of correlations are small as well, the determinants of the variance-covariance matrices of the corrected correlations were often extremely tiny (<10'10) , especially for the cases when p = 500. For this reason, after these determinants were 3 before counts and calculated, they were multiplied by p comparisons among the methods were made. Without this convention, the majority of the cases would have shown all 2000 replications to have "improper" variance-covariance matrices. Results from Count Data Qorrected cprrelations greater than upity. Here, a "case" refers to one of the seven different population correlation sets from Table 1. The first count examined was the percent of corrected correlations greater than unity, for each case and method of correcting the correlations. In general, the frequency of corrected correlations greater than unity, as shown in Table 2, was small (no more than three percent). In two of the cases ( p. = (.00, .00, .00) and p = (.40. than unit the cases the numbe tiny (les produce c univariat was 50, a overcorre (and larg (1975) an Corrected As t imPrOper I 57 p = (.40, .30, .10)), no corrected correlations were greater than unity for any sample size or reliability combination. In the cases where p = (.60, .40, .20) and p = (.60, .40, -.20), the number of corrected correlations greater than unity was tiny (less than .11%). In these cases, the only methods to produce corrected correlations greater than unity were the univariate and the Gleser corrections, when the sample size was 50, and the reliability triple was (.70, .70, .70). The overcorrected correlations occurred for the first correlation (and largest) in the triple (.60). The Book and Petersen (1975) and.Fuller and Hidiroglou (1978) corrections never gave corrected correlations greater than unity in any case. As the population correlations increased, the percent of improper corrected correlations increased. In the case where p = (.40, .30, .70) and p = 50, for the reliability triple (.70, .70, .70) both.the univariate and.Gleser corrections had 0.75% (15 out of 2000) invalid corrected correlations. In the same case, but where the reliability triple (.90, .80, .70) was used, the percents were 0.20% and 0.15% for the univariate and Gleser methods, respectively. All of the invalid corrected. correlations occurred.*when. the largest (third) population correlation (p = .70) was corrected. In fact, in all cases, no sample estimate of a population correlation less than .60 gave a corrected correlation greater than unity. (able 2 Percent of CC’ 0 g .7 C : ”MW 0.C Bock 0.0: Fuller 0.0: Gleser 0% + = .85 C2; 5°“ 0.0c FUller 0.0C Blescr 00C £1" 9 92: = my” 0.00 3°“ 0.00 FUller .00 Gleser 0'00 ‘31. = 1.00 02, Um" 0.0c Bock 0.0C FUller _OC Gleser Table 2 Percggt of Corrected Correlations Greater than Unitx Case 1. 012 3 .0 013 3 .0 023 = .0 Pop. Corr. 0.00 011 "‘ -7 922 ‘-7 Univar Bock Fuller Gleser 0.00 0.00 0.00 0.00 n=50 0.00 0.00 933 3 -7 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 011 = .85 022 8.85 033 3 .85 Univar Bock Fuller Gleser 0.00 0.00 0.00 0.00 911 3 -9 922 “5 Univar Bock Fuller Gleser p11 3 1.00 Univar Bock Fuller Gleser 0.00 0.00 0.00 0.00 022 3 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 033 3 0.00 0.00 0.00 0.00 1.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 .7 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 1.00 0.00 0.00 0.00 0.00 n=100 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 58 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 n=250 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 n=500 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 59 Table 2 (Cont’d) C358 2. 012 3 .4 913 3 .3 023 3 .1 n=50 n=100 n=250 n=500 Pop. Corr. 0.40 0.30 0.10 0.40 0.30 0.10 0.40 0.30 0.10 0.40 0.30 0.10 911 = -7 922 3-7 933 = -7 Univar 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Bock 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Fuller 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Gleser 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 pm 3 .85 022 3.85 p33 3 .85 Univar 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Back 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Fuller 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Gleser 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 pm 3 .9 p22 3.8 p33 3 .T' Univar 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Bock 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Fuller 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Gleser 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 pm 3 1.00 022 3 1.00 033 3 1.00 Univar 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Bock 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Fuller 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Gleser 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 60 Table 2 (Cont'd) l V Case 3. p12 = .4 p13 = .3 p23 - n=50 n=100 n=250 n=500 Pop. Corr. 0.40 0.30 0.70 0.40 0.30 0.70 0.40 0.30 0.70 0.40 0.30 0.70 lon = -7 922 ‘-7 933 g -7 Univar 0.00 0.00 0.75 Back 0.00 0.00 0.00 Fuller 0.00 0.00 0.00 Gleser 0.00 0.00 '0.75 .00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 .00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 .00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 .00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0000 l' I m U' p11 3 .85 p22 3.85 p33 Univar 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Bock 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Fuller 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Gleser 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 p“ 3 .9 p22 3.8 p33 3 .7 Univar 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Bock 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Fuller 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Gleser 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 (011 3 1.00 p22 3 1.00 033 3 1.00 Univar 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Bock 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Fuller 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Gleser 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Table 2 (Cont'd) Case 4. p12 = .6 p13 = .4 023 = .2 Pop. Corr. 0.60 n=50 0.40 0.20 lon = :7 922 ‘-7 933 = -7 Univar Bock Fuller Gleser l911 ‘ ~55 922 ‘-35 033 Univar Bock Fuller Gleser 911 3 -9 022 3-3 Univar Bock Fuller Gleser p11 a 1.00 Univar Bock Fuller Gleser 0.05 0.00 0.00 0.05 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 922 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 = .85 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 933 = -7 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 1.00 033 3 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.60 .00 .00 .00 .00 000° .00 .00 .00 .00 0000 0.00 0.00 0.00 0.00 1.00 0.00 0.00 0.00 0.00 n=100 0.40 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 61 0.20 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.60 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 n=250 0.40 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.20 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.60 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 n=500 0.40 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.20 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Table 2 (Cont'd) Case 5. p12 = .6 p13 = .4 p23 =-.2 Pop. Corr. 0.60 911 = -7 022 =-7 Univar Bock Fuller Gleser 0.10 0.00 0.00 0.10 n=50 0.40 l933 ‘ 0.00 0.00 0.00 0.00 l’11 3 -35 022 “~55 933 Univar Bock Fuller Gleser 0.00 0.00 0.00 0.00 911 a -9 022 “3 Univar Bock Fuller Gleser p11 . 1.00 Univar Bock Fuller Gleser 0.00 0.00 0.00 0.00 022 ‘ 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 033 = 0.00 0.00 0.00 0.00 1.00 0.00 0.00 0.00 0.00 -0.20 0.00 0.00 0.00 0.00 .85 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 033 ' 0.00 0.00 0.00 0.00 0.60 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 1.00 0.00 0.00 0.00 0.00 n=100 0.40 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 62 -0.20 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.60 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 n=250 0.40 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 -0.20 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.60 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 n=500 0.40 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 -0.20 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Table 2 (Cont'd) case 6. 012 3 .7 013 3 .6 023 3 Pop. Corr. 0.70 1911 3 -7 l"22 5-7 Univar Bock Fuller Gleser 0.90 0.00 0.00 0.75 n=50 0.60 933 = 0.15 0.00 0.00 0.15 0.10 .7 0.00 0.00 0.00 0.00 011 3 .85 022 3.85 033 3 .85 Univar Bock Fuller Gleser 0.00 0.00 0.00 0.00 011 3 -9 922 3-3 Univar Bock Fuller Gleser 011 3 1.00 Univar Bock Fuller Gleser 0.00 0.00 0.00 0.00 022 3 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 p33: 0.00 0.00 0.00 0.00 1.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 .7 0.00 0.00 0.00 0.00 933 = 0.00 0.00 0.00 0.00 I I _l 0.70 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 n=100 0.60 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 63 0.10 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.70 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 n=250 0.60 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.10 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.70 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 n=500 0.60 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.10 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Table 2 (Cont'd) Case 7. 012 = .7 013 = .6 023 - Pop. Corr. 0.70 n=50 0.60 1011 = -7 922 3-7 933 = Univar laock Fuller (Sleser 1011 3 -35 022 =-35 933 ‘Jnivar laock Fuller Gleser 1.15 0.00 0.00 1.15 0.00 0.00 0.00 0.00 On 3 .9 022 3.8 lJnivar Eiock Fuller Gleser 011 3 1.00 Univar lilock F ul ler (Sileser 0.00 0.00 0.00 0.00 922 = 0.00 0.00 0.00 0.00 0.15 0.00 0.00 0.30 0.00 0.00 0.00 0.00 033 3 0.00 0.00 0.00 0.00 1.00 0.00 0.00 0.00 0.00 0.80 .7 3.00 0.00 0.00 2.95 .85 0.05 0.00 0.00 0.05 1.65 0.00 0.00 1.60 0.00 0.00 0.00 0.00 0.70 .00 .00 .00 .00 0000 0.00 0.00 0.00 0 0.00 0.00 0.00 0.00 .00 0.00 0.00 0.00 0.00 n=100 0.60 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 64 0.80 0.40 0.00 0.00 0.60 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.70 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 n=250 0.60 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.80 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.70 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 n=500 0.60 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.80 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 65 In the case p = (.70, .60, .10), the results were similar to the case mentioned previously. For _I_1_ = 50, 0.90% of the corrected correlations from the univariate and 0.75% for the Gleser methods gave improper corrected correlations, for the first correlation in the triple (.70) . The second correlation in the triple (.60) yielded .15% corrected correlations greater than unity for both the univariate and Gleser cases with reliabilities equal to .70 and sample size of 50. No other reliability and sample size combinations produced invalid corrections for this case. The case which showed the highest percent of invalid corrected correlations was p = (.70, .60, .80), as expected. 'The univariate method produced 1.15% improper corrected «correlations for the first element in the triple (.70), 0.15% for the second element (.60), and 3.00% for the third element (.80) for n = 50 and the reliability triple (.70, .70, .70). 'These numbers were 0.95%, 0.30%, and 2.95% respectively for the Gleser corrected correlations for the same combination. fIhe third correlation in the triple (.80) also yielded non- :zero percents for the cases where reliability triples were (.85, .85, .85) and (.90, .80, .70) with g = 50 reliabilities (.70, .70, .70) with Q = 100. For the univariate correction tzhese percents ranged from 0.05% to 1.45% and for the Gleser <:orrection, from 0.05% to 1.60%. Overall, in no case where the sample size was 250 or 500, or the reliabilities were 1.0, did any invalid corrected 66 correlations occur. These results show that where univariate corrections are applied, for moderate to large correlations with small sample sizes and somewhat lower reliabilities, the chance of corrected correlations greater than one is non-zero. Because the Bock and.Petersen (1975) and.Fuller and.Hidiroglou (1978) corrections adjust for these problems, their use may be ‘warranted when such problems are anticipated. Determinants of the corrected correlation.matrices. The vast majority of the matrices of the corrected correlations were positive definite. Table 3 displays, for each method, the percentages of the corrected correlation matrices that were less than or equal to 0. The cases, =(.40, .30, .10) and p = (.00, .00, .00), did.not produce any invalid corrected correlation matrices and the case p = (.60, .40, .20) produced only a few invalid matrices, as shown in the table. For the case p = (.60, .40, -.20) with reliability triple (.70, .70, .70) the univariate, Bock and Petersen (1975), and Gleser (1992) methods produced 7.2%, 2.15%, and 7.35% improper corrected correlation matrices, respectively. These percentages declined to 1.65%, 0.05%, and 1.55% when, for the same reliability triple, g = 100. With the (.85, .85, .85) reliability triple and.Q = 50, the univariate and Gleser methods gave 0.15% invalid matrices, while the Bock and Petersen method gave 0.05%. Finally, for the reliability triple (.90, .80, .70) and g = 50, the percentages for the 67 univariate, Book and Petersen, and Gleser methods were 0.55%, 0.25%, and 0.60%, respectively. Table 3 Percgpt of Determinants of Corrected Correlation figtrices Less than or Egual to Zero (2000 replications) Case 1. 912 3 .0 p13 3 .0 023 3 .0 n=50 n=100 n=250 n=500 911 = -7 022 =-7 933 = -7 Univar 0.00 0.00 0.00 0.00 Bock 0.05 0.00 0.00 0.00 Fuller 0.00 0.00 0.00 0.00 Gleser 0.00 0.00 0.00 0.00 pm 3 .85 922 3.85 033 3 .85 Univar 0.00 0.00 0.00 0.00 Bock 0.00 0.00 0.00 0.00 Fuller 0.00 0.00 0.00 0.00 Gleser 0.00 0.00 0.00 0.00 911 = -9 922 3-3 033 3 -7 Univar 0.00 0.00 0.00 0.00 Bock 0.00 0.00 0.00 0.00 Fuller 0.00 0.00 0.00 0.00 Gleser 0.00 0.00 0.00 0.00 £11 3 1.00 022 3 1.00 033 3 1.00 Univar 0.00 0.00 0.00 0.00 Bock 0.00 0.00 0.00 0.00 Fuller 0.00 0.00 0.00 0.00 Gleser 0.00 0.00 0.00 0.00 68 Table 3 (Cont'd) Case 2. 012 3 .4 913 3 .3 023 3 .1 n=50 n=100 =250 n=500 p11 3 -7 022 3-7 933 3 -7 Univar 0.00 0.00 0.00 0.00 Bock 0.00 0.00 0.00 0.00 Fuller 0.00 0.00 0.00 0.00 Gleser 0.00 0.00 0.00 0.00 p11 3 .85 022 3.85 033 3 .85 Univar 0.00 0.00 0.00 0.00 Bock 0.00 0.00 0.00 0.00 Fuller 0.00 0.00 0.00 0.00 Gleser 0.00 0.00 0.00 0.00 1011 3 -9 922 3-8 033 3 -7 Univar 0.00 0.00 0.00 0.00 Bock 0.00 0.00 0.00 0.00 Fuller 0.00 0.00 0.00 0.00 Gleser 0.00 0.00 0.00 0.00 pm 3 1.00 022 3 1.00 033 3 1.00 Univar 0.00 0.00 0.00 0.00 Bock 0.00 0.00 0.00 0.00 Fuller 0.00 0.00 0.00 0.00 Gleser 0.00 0.00 0.00 0.00 69 Table 3 (Cont'd) n=50 n=100 =250 n=500 Univar 1.60 0.00 0.00 0.00 Bock 0.80 0.05 0.00 0.00 Fuller 0.00 0.00 0.00 0.00 Gleser 1.55 0.00 0.00 0.00 pm 3 .85 022 3.85 p33 3 .85 Univar 0.00 0.00 0.00 0.00 Bock 0.00 0.00 0.00 0.00 Fuller 0.00 0.00 0.00 0.00 Gleser 0.00 0.00 0.00 0.00 911 3 -9 922 3-8 033 3 -7 Univar 0.30 0.00 0.00 0.00 Back 0.30 0.00 0.00 0.00 Fuller 0.00 0.00 0.00 0.00 Gleser 0.30 0.00 0.00 0.00 pm 3 1.00 022 3 1.00 [’33 3 1.00 Univar 0.00 0.00 0.00 0.00 Back 0.00 0.00 0.00 0.00 Fuller 0.00 0.00 0.00 0.00 Gleser 0.00 0.00 0.00 0.00 70 Table 3 (Cont'd) Case 4. 012 3 .6 013 3 .4 923 3 .2 n=50 n=100 n=250 n=500 Univar 0.40 0.00 0.00 0.00 Bock 0.50 0.00 0.00 0.00 Fuller 0.00 0.00 0.00 0.00 Gleser 0.45 0.00 0.00 0.00 011 3 .85 p22 3.85 p33 3 .85 Univar 0.00 0.00 0.00 0.00 Bock 0.00 0.00 0.00 0.00 Fuller 0.00 0.00 0.00 0.00 Gleser 0.00 0.00 0.00 0.00 1911 3 -9 022 3-8 033 3 -7 Univar 0.00 0.00 0.00 0.00 Back 0.05 0.00 0.00 0.00 fuller 0.00 0.00 0.00 0.00 Gleser 0.00 0.00 0.00 0.00 Du 3 1.00 022 3 1.00 033 3 1.00 Univar 0.00 0.00 0.00 0.00 Bock 0.00 0.00 0.00 0.00 Fuller 0.00 0.00 0.00 0.00 Gleser 0.00 0.00 0.00 0.00 71 Table 3 (Cont'd) Case 5. 012 3 .6 913 3 .4 023 3'.2 n=50 n=100 n=250 n=500 911 3 -7 022 3-7 933 3 -7 Univar 7.20 1.65 0.00 0.00 Bock 2.15 0.05 0.00 0.00 Fuller 0.00 0.00 0.00 0.00 Gleser 7.35 1.55 0.00 0.00 On 3 .85 p22 3.85 p33 3 .85 Univar 0.15 0.00 0.00 0.00 Bock 0.05 0.00 0.00 0.00 Fuller 0.00 0.00 0.00 0.00 Gleser 0.15 0.00 0.00 0.00 911 3 ~9 922 3-8 p33 3 -7 Univar 0.55 0.00 0.00 0.00 Bock 0.25 0.00 0.00 0.00 Fuller 0.00 0.00 0.00 0.00 Gleser 0.60 0.00 0.00 0.00 pH 3 1.00 022 3 1.00 p33 3 1.00 Univar 0.00 0.00 0.00 0.00 Bock 0.00 0.00 0.00 0.00 Fuller 0.00 0.00 0.00 0.00 Gleser 0.00 0.00 0.00 0.00 72 Table 3 (Cont'd) n=50 n=100 n=250 n=500 Univar 14.65 3.85 0.20 0.00 Bock 5.70 0.45 0.00 0.00 Fuller 0.00 0.00 0.00 0.00 Gleser 14.85 3.80 0.20 0.00 ,011 3 .85 022 3.85 p33 3 .85 Univar 1.75 0.00 0.00 0.00 Bock 0.45 0.00 0.00 0.00 Fuller 0.00 0.00 0.00 0.00 Gleser 1.80 0.00 0.00 0.00 911 3 -9 022 3-8 933 3 -7 Univar 1.95 0.10 0.00 0.00 Bock 0.85 0.00 0.00 0.00 Fuller 0.00 0.00 0.00 0.00 Gleser 2.25 0.05 0.00 0.00 011 3 1.00 [322 3 1.00 p33 3 1.00 Univar 0.00 0.00 0.00 0.00 Bock 0.00 0.00 0.00 0.00 Fuller 0.00 0.00 0.00 0.00 Gleser 0.00 0.00 0.00 0.00 73 Table 3 (Cont'd) Case 7. 912 3 .7 D13 3 .6 023 3 .8 n=50 n=100 n=250 n=500 011 3 -7 922 3-7 p33 3 -7 Univar 8.20 1.00 0.00 0.00 Bock 4.60 0.25 0.00 0.00 Fuller 0.00 0.00 0.00 0.00 Gleser 8.25 1.10 0.00 0.00 pm 3 .85 022 3.85 D33 3 .85 Univar 0.15 0.00 0.00 0.00 Back 0.10 0.00 0.00 0.00 Fuller 0.00 0.00 0.00 0.00 Gleser 0.10 0.00 0.00 0.00 911 3 ~9 022 3-8 933 3 -7 Univar 2.40 0.05 0.00 0.00 Bock 0.95 0.05 0.00 0.00 Fuller 0.00 0.00 0.00 0.00 Gleser 2.65 0.05 0.00 0.00 011 3 1.00 022 3 1.00 033 3 1.00 Univar 0.00 0.00 0.00 0.00 Back 0.00 0.00 0.00 0.00 Fuller 0.00 0.00 0.00 0.00 Gleser 0.00 0.00 0.00 0.00 For the case p = (.40, .30, .70) a smaller number of invalid corrected correlation matrices were found. Again, the reliability triple (.70, .70, .70) in combination with g = 50 produced the most problems, with 1.60%, 0.80%, and 1.55% invalid matrices for the univariate, Bock and Petersen, and Gleser methods respectively. The Book and Petersen method also produced one (0.05%) invalid matrix for the same reliability combination with a sample size n = 100. For the 74 reliability triple (.90, .80, .70) with Q = 50, each of the three methods mentioned above produce 0.30% improper corrected correlation matrices. The case with the greatest number of invalid corrected correlation matrices was p = (.70, .60, .10). Nearly 15% of the matrices were invalid for the reliability triple (.70, .70, .70) and g = 50 case for the univariate and Gleser methods. The Bock and Petersen method in this same combination produced 5.70% invalid matrices. Within the same reliability triple, but with n = 100, the univariate, Bock and Petersen and Gleser method produced 3.85%, 0.45%, and 3.80% invalid corrected correlation matrices respectivelyu This particular correlation triple was the only one to produce invalid results when the sample size was 250. With this sample size and the reliability triple (.70, .70, .70), the univariate and Gleser methods yielded 0.20% invalid matrices. When the (.85, .85, .85) reliability triple was used, somewhat fewer problems were found. With this triple and n = 50, the univariate, Bock and Petersen, and Gleser methods produced invalid matrices 1.75%, 0.45%, and 1.80% of the time, respectively. When the reliability values were changed to (.90, .80, .70), the three methods gave 1.95%, 0.85%, and 2.25% invalid matrices for _n = 50. With this same reliability triple, but with g = 100, the univariate and Gleser methods gave 0.10% and 0.05% improper matrices, respectively. 75 In the last case, p =(.70, .60, .80), with reliability triple (.70, .70, .70) and g_= 50, the percentages of improper 1corrected,correlationumatrices were 8.20%, 4.60% and 8.25% for the univariate, Bock and Petersen, and Gleser methods, respectively. With E = 100 and the same reliability values, these percentages changed to 1.00%, 0.25%, and 1.10%, respectively. This case also produced a small number of invalid matrices for the (.85, .85, .85) reliability triple with n = 50. These numbers were 0.15% (univariate), 0.10% (Bock and Petersen), and 0.10% (Gleser). The reliability triple (.90, .80, .70) yielded 2.40% invalid matrices for the univariate method, 0.95% for the Bock and Petersen method, and 2.65% for the Gleser method when the sample size was 50. With the same reliability triple and _r; = 100, all three of the above methods gave 0.05% improper matrices, or one matrix out of 2000. These results indicate that the Fuller and Hidiroglou method does prevent improper corrected correlation matrices. None of the combinations, when the Fuller and Hidiroglou method was used, produced improper results. Problems with the determinants of the corrected correlation matrices appeared to be related in part to the presence of corrected correlations larger than unity. However, invalid matrices occur more frequently than correlations larger than one. This result indicates that other problems result from these corrections, which would imply that the Stanley and Wang (1969) inequality 76 is being violated in situations other than when correlations greater than unity occur. In no case did the original sample correlation matrix have a determinant less than 0. Therefore, the problems occurred after the correction had been made. Determinants of variance-covariance matrices of the corrected correlations. Also recorded were the percentages of invalid variance-covariance matrices of the corrected correlations for each method. Table 4 displays these results. The case p =(.40, .30, .10) did not produce any invalid matrices, while the case p= (.00, .00, .00) produced. 1 problematic matrix, for the Book and Petersen correction with reliability triple (.70, .70, .70) and Q = 50. As with the results for the corrected correlations greater than unity and the number of improper corrected correlation matrices, the majority of the combinations that caused problems had small sample sizes and reliability values. 77 Table 4 Pergggt of Determinants of Varigpce/Covarigpce Matrices Legs than or Equal to Zero (2000 replications} Case 1. 012 3 .0 013 3 .0 023 3 .0 n=50 n=100 n=250 n=500 011 3 -7 922 3-7 033 3 -7 Univariate 0.00 0.00 0.00 0.00 Bock 8 Petersen 0.00 0.00 0.00 0.00 Fuller 8 Hidiroglou 0.00 0.00 0.00 0.00 Large Sample 0.00 0.00 0.00 0.00 Fixed Reliability 0.00 0.00 0.00 0.00 On 3 .85 922 3.85 033 3 .85 Univariate 0.00 0.00 0.00 0.00 Bock 8 Petersen 0.00 0.00 0.00 0.00 Fuller 8 Hidiroglou 0.00 0.00 0.00 0.00 Large Sample 0.00 0.00 0.00 0.00 Fixed Reliability 0.00 0.00 0.00 0.00 911 3 -9 922 3-3 033 3 -7 Univariate 0.00 0.00 0.00 0.00 Bock 8 Petersen 0.00 0.00 0.00 0.00 Fuller 8 Hidiroglou 0.00 0.00 0.00 0.00 Large Sample 0.00 0.00 0.00 0.00 Fixed Reliability 0.00 0.00 0.00 0.00 0113 1.00 ,022 3 1.00 p33 3 1.00 Univariate 0.00 0.00 0.00 0.00 Bock 8 Petersen 0.00 0.00 0.00 0.00 Fuller 8 Hidiroglou 0.00 0.00 0.00 0.00 Large Sample 0.00 0.00 0.00 0.00 Fixed Reliability 0.00 0.00 0.00 0.00 Table 4 (Cont'd) Case 2. p12 3 .10 013 3 .3 023 3 . n=50 911 3 -7 p22 3-7 933 3 -7 Univariate Beck 8 Petersen Fuller 8 Hidiroglou Large Sample Fixed Reliability pm 3 .85 022 3.85 p33 3 .85 Univariate Bock 8 Petersen Fuller 8 Hidiroglou Large Sample Fixed Reliability 911 3 -9 022 3-5 933 3 -7 Univariate Back 8 Petersen Fuller 8 Hidiroglou Large Sample Fixed Reliability p11 3 1.00 022 3 1.00 [333 3 1.00 Univariate Bock 8 Petersen Fuller 8 Hidiroglou Large Sample Fixed Reliability n=100 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 78 n=250 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 n=500 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Table 4 (Cont'd) Univariate Bock 8 Petersen Fuller 8 Hidiroglou Large Sample Fixed Reliability On 3 .85 [222 3.85 p33 3 .85 Univariate Bock 8 Petersen Fuller 8 Hidiroglou Large Sample Fixed Reliability 011 3 -9 022 3-3 033 3 -7 Univariate Beck 8 Petersen Fuller 8 Hidiroglou Large Sample Fixed Reliability D11 3 1.00 022 3 1.00 p33 3 1.00 Univariate Bock 8 Petersen Fuller 8 Hidiroglou Large Sample Fixed Reliability n=50 0.95 1.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.20 0.30 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 79 n=100 0.00 0.05 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 n=250 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 n=500 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Table 4 (Cont'd) Case 6. p12 3 .6 013 3 .4 p23 3 . 011 3 -7 922 3-7 033 3 -7 Univariate Bock 8 Petersen Fuller 8 Hidiroglou Large Sample Fixed Reliability 011 3 .85 022 3.85 p33 3 .85 Univariate Back 8 Petersen Fuller 8 Hidiroglou Large Sample Fixed Reliability 911 3 -9 922 3-3 033 3 -7 Univariate Back 8 Petersen Fuller 8 Hidiroglou Large Sample Fixed Reliability 0118 1.00 p22 3 1.00 933 3 1.00 Univariate Bock 8 Petersen Fuller 8 Hidiroglou Large Sample Fixed Reliability n=50 0.10 0.55 0.00 0.00 0.00 0.00 0.05 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 80 n=100 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 n=250 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 n=500 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Table 4 (Cont'd) Case 5. 012 3 .6 013 3 .4 923 3 3.2 n=50 011 3 -7 922 3-7 933 3 -7 Univariate Bock 8 Petersen Fuller 8 Hidiroglou Large Sample Fixed Reliability p11 3 .85 p22 3.85 D33 3 .85 Univariate Bock 8 Petersen Fuller 8 Hidiroglou Large Sample Fixed Reliability 011 3 ~9 922 3-3 933 3 '7 Univariate Back 8 Petersen Fuller 8 Hidiroglou Large Sample Fixed Reliability p11 3 1.00 022 3 1.00 ‘73:; 3 1.00 Univariate Back 8 Petersen Fuller 8 Hidiroglou Large Sample Fixed Reliability n=100 0.25 3.25 0.00 .00 .10 .00 .00 .00 00°00 .00 .35 .00 .00 .00 00°00 0.00 0.00 0.00 0.00 0.00 81 n=250 0.05 0.15 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 n=500 0.00 0.00 .00 00 .00 .00 .00 00°00 .00 .00 .00 .00 OOOOO 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Table 4 (Cont'd) Case 6. 012 = .7 913 = .6 p23 = . p11 3 -7 022 3-7 033 3 -7 Univariate Bock 8 Petersen Fuller 8 Hidiroglou Large Sample Fixed Reliability p11 3 .85 p22 3.85 p33 3 .85 Univariate Bock 8 Petersen Fuller 8 Hidiroglou Large Sample Fixed Reliability p11 3 -9 022 3-3 933 3 -7 Univariate Bock 8 Petersen Fuller 8 Hidiroglou Large Sample Fixed Reliability 01131.00 022 31.00 p33 3 1.00 univariate Bock 8 Petersen Fuller 8 Hidiroglou Large Sample Fixed Reliability n=50 2.40 8.05 0.00 0.00 0.00 0.15 0.65 0.00 0.00 0.00 0.05 0.95 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 82 n=100 0.05 0.90 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.05 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 n=250 0.05 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 n=500 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 table 4 (Cont'd) Case 7. 012 3 .7 013 3 .6 923 3 .8 p11 3 -7 922 3-7 933 3 -7 Univariate Bock 8 Petersen Fuller 8 Hidiroglou Large Sample Fixed Reliability 011 3 .85 022 3.85 033 3 .85 Univariate Bock 8 Petersen Fuller 8 Hidiroglou Large Sample Fixed Reliability 911- 3 -9 922 3-8 033 3 -7 Univariate Bock 8 Petersen Fuller 8 Hidiroglou Large Sample Fixed Reliability p11 3 1.00 p22 3 1.00 p33 3 1.00 Univariate Bock 8 Petersen Fuller 8 Hidiroglou large Sample Fixed Reliability n=50 7.00 5.25 0.00 0.00 0.00 0.10 0.20 0.00 0.00 0.00 2.55 1.55 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 83 n=100 1.10 0.25 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.05 0.05 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 n=250 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 n=500 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 84 For the case where p = (.60, .40, .20) the Bock and Petersen method produced 0.55% and 0.05% invalid matrices when a = 50 and the reliability triples (.70, .70, .70) and (.85, .85, .85) were used, respectively. For the first reliability triple, the univariate correction yielded 0.10% invalid variance-covariance matrices. The rest of the combinations for this case did not provide any problematic matrices. When the p = (.60, .40, -.20) case was examined, again the Bock and Petersen and univariate methods were the only ones to produce invalid matrices. For the reliability triple (.70, .70, .70) and n = 50, the univariate and Bock and Petersen methods produced 0.25% and 3.25% improper matrices respectivelyu These percentages were reduced.when.n_= 100 and the percentages were also non-zero when other reliability triples were used. For the case where p = (.40, .30, .70) with reliability triple (.70, .70, .70) the only combinations with notable results were the univariate method, which gave 0.95% invalid variance-covariance matrices, and the Bock and Petersen method 'which showed 1.00% invalid matrices for n = 50. When p = (.70, .60, .10) the univariate and Bock and Fhetersen methods again produced invalid variance-covariance matrices, particularly in the case where the reliability tariple (.70, .70, .70) and the sample size n = 50 was used. II: that case, the univariate method produced 2.40% improper 85 matrices, while the Book and Petersen gave 8.05%. Other reliability and sample size combinations produced non-zero percentages smaller than this case, as shown in the table. Finally, the last case, p == (.70, .60, .80) produced improper matrices for the reliability triples (.70, .70, .70), (.85, .85, .85) and (.90, .80, .70) for n = 50. The rates for 0 these three cases for the univariate method were 7.00%, 0.10% and 2.55% respectively. For the Bock and Petersen method, these rates were 5.25%, 0.20%, and 1.55%, respectivelyn ‘When n =100, the reliability triples (.70, .70, .70) and (.90, .80, .70) also yielded non-zero percents of invalid matrices. For the former triple, the rates were 1.10% for the univariate method, and 0.25% for the Bock and Petersen method, while for the latter triple, the rates for these methods were both 0.05%. The size of the determinants of the variance-covariance matrices. Besides counting the variance-covariance matrices of the corrected correlations that were non-positive definite, the size of the determinant of each matrix was also examined. Table 5 displays the mean determinants for each case and lcombination, for the 2000 replications. The means displayed :Ln.the tables are actually the mean determinants multiplied.by Q3. As shown in the tables, the univariate and Bock and ertersen methods produced the smallest determinants. These WEere considerably smaller for several of the cases. For all 013 the cases, when the reliability triple (1.00, 1.00, 1.00) 86 was used, the determinants were virtually identical for each method. These results fit with the number of improper variance-covariance matrices found above. The methods and combinations which yielded the smallest determinants also produced the highest numbers of invalid variance-covariance matrices. Table 5 rmi t f ri nc - ov riance Hatric ES2D_22E3___nl0L3_2L_I2__1L_JL£L_£L_____________£! of the ggrrggtgg ggrpglagions 52000 rgglicatigggz gase 1. 012 8 .0 013 8 .0 023 = .0 911 3 -7 022 3-7 933 3 Univar Bock Fuller Large Sample Fixed Rel. n=50 0.6909 0.5827 7.6797 7.6797 7.7899 n=100 .7 0.8336 0.7561 8.0657 8.0657 8.2097 011 3 .85 022 3.85 033 3 .85 Univar Bock Fuller Large Sample Fixed Rel. 0.7777 0.6365 2.2706 2.2706 2.3159 0.8829 0.7885 2.6660 2.6660 2.6678 911 3 -9 922 3-3 933 3 -7 Univar Bock Fuller Large Sample Fixed Rel. 0n = 1.00 Univar Bock Fuller Large Sample Fixed Rel. 0.7662 0.6650 3.6131 3.6131 3.5079 0.8352 0.6821 0.8357 0.8357 0.8358 0.8669 0.6388 3.6518 3.6518 3.7015 022 3 1.00 033 = 1.00 0.9131 0.8166 0.9137 0.9137 0.9137 n=250 0.9280 0.8911 8.2551 8.2551 8.3239 0.9512 0.9072 2.5716 2.5716 2.5816' 0.9638 0.8123 3.8196 3.8196 3.8601 0.9635 0.9218 0.9661 0.9661 0.9661 n=500 0.9629 0.9628 8.3875 8.3875 8.6231 0.9757 0.9536 2.6130 2.6130 2.6180 0.9730 0.8980 3.8720 3.8720 3.8819 0.9826 0.9593 0.9830 0.9830 0.9830 table 5 (Cont'd) Cgse Z: [312 3 .4 013 3 0113 -7 922 3-7 933 3 Univar Bock Fuller Large Sample Fixed Rel. n=50 0.3587 0.2863 5.2766 5.2766 5.8371 .3 023 3 n=100 .7 0.3956 0.3637 5.1876 5.1876 5.7006 011 3 .85 022 3.85 033 3 .85 Univar Bock Fuller Large Sample Fixed Rel. 011 3 .9 922 8. Univar Bock Fuller Large Sample Fixed Rel. 0n a 1.00 Univar Bock Fuller Large Sample Fixed Rel. 0.3863 0.2961 1.3273 1.3273 1.6263 8 0333. 0.3853 0.1509 2.1002 2.1002 2.2710 0.6056 0.3076 0.6057 0.6057 0.6058 0.6156 0.3597 1.3656 1.3656 1.6588 7 0.6128 0.1518 2.0806 2.0806 2.2371 022 s 1.00 033 . 1.00 0.6266 0.3636 0.6269 0.6269 0.6269 .1 n=250 0.6253 0.3920 5.2869 5.2869 5.7761 0.6336 0.6080 1.3877 1.3877 1.6793 0.6336 0.1673 2.1233 2.1233 2.2762 0.6373 0.6110 0.6376 0.6376 0.6376 NNNOO fi-l-‘OO U'IU'IWOO OOOOO =500 .6368 .6188 .3285 .3285 .8119 .6389 .6198 .3969 .3969 .6883 .6386 .1661 .1252 .1252 .2769 .6380 .6261 .6383 .6383 .6383 Table 5 (Cont'd) $322.2; 912 3 .6 013 3 0n 3 -7 022 3-7 033 3 Univar Bock Fuller Large Sample Fixed Rel. n-50 0.0863 0.0633 2.5611 2.5667 3.1538 p11 3 .85 022 3.85 p33 3 .85 Univar Bock Fuller Large Sample Fixed Rel. 0.0833 0.0655 0.6692 0.6692 0.5601 0.0797 0.0696 0.6158 0.6158 0.5038 011 3 .9 022 3.8 D33 3 .7 Univar Bock Fuller Large Sample Fixed Rel. On 3 1.00 Univar Bock Fuller Large Sample Fixed Rel. 0.0907 0.0261 0.9688 0.9689 1.1539 0.0832 0.0186 0.8756 0.8756 1.0799 022 31.00 033 31.00 0.0807 .3 923 3 .7 n=100 n=250 .7 0.0816 0.0776 0.0716 0.0761 2.6373 2.2706 2.6373 2.2706 3.0535 2.8715 0.0786 0.0778 0.6066 0.6066 0.6918 0.0791 0.0162 0.8290 0.8290 1.0313 0.0771 0.0765 0.0772 0.0772 0.0772 0.0771 0.0761 0.0772 0.0772 0.0772 89 Table 5 (Cont'd) 9118—3; 912 3 -5 p13 3 011 3 -7 922 3-7 933 3 Univar Bock Fuller Large Santa Fixed Rel. n-SO 0.1689 0.1171 3.6035 3.6056 6.0520 Du 3 .85 022 3.85 033 3 .85 Univar lock Fuller Large Sample Fixed Rel. 911 3 -9 022 3-3 933 3 Univar lock Fuller Large Sample Fixed Rel. p11 3 1.00 Univar Bock Fuller Large Sample Fixed Rel. 0.1555 0.1222 0.6819 0.6819 0.7831 0.1666 0.0665 1.0619 1.0619 1.2167 022 3 1.00 0.1572 0.1216 0.1573 0.1573 0.1573 .4. p23 . .2 n8100 n=250 .7 0.1566 0.1521 0.1358 0.1667 3.2531 3.1028 3.2531 3.1028 3.8803 3.7250 0.1531 0.1562 0.1382 0.1657 0.6528 0.6669 0.6528 0.6669 0.7529 0.7651 .7 0.1535 0.1500 0.0396 0.0359 1.0636 0.9910 1.0636 0.9910 1.2159 1.1602 033 31.00 0.1522 0.1551 0.1352 0.1671 0.1523 0.1553 0.1523 0.1553 0.1526 0.1553 Table 5 (Cont'd) 319.5. 012 3 .6 013 3 .6 023 3 3-2 n-50 n=100 n=250 0n 3 .7 022 3.7 033 3 .7 Univar 0.0652 0.0616 0.0572 lock 0.0651 0.0676 0.0520 Fuller 2.5025 2.3815 2.3277 Large Sample 2.5268 2.3860 2.3277 Fixed Rel. 3.2051 3.0605 3.0126 011 3 .85 022 3.85 033 3 .85 Univar 0.0561 0.0553 0.0567 Bock 0.0633 0.0695 0.0526 Fuller 0.3682 0.3680 0.3576 Large Sample 0.3682 0.3680 0.3576 Fixed Rel. 0.6685 0.6699 0.6583 011 3 .9 022 3.8 033 3 .7 Univar 0.0559 0.0533 0.0550 Bock 0.0158 0.0157 0.0157 Fuller 0.6635 0.6238 0.6200 Large Sample 0.6637 0.6238 0.6200 Fixed Rel. 0.8687 0.8075 0.8032 01131.00 022 31.00 033 31.00 Univar 0.0532 0.0566 0.0567 Bock 0.0655 0.0691 0.0525 Fuller 0.0532 0.0567 0.0567 Large Sample 0.0532 0.0567 0.0567 Fixed Rel. 0.0532 0.0567 0.0568 n=500 0.0570 0.0528 2.3139 2.3139 2.9989 0.0553 0.0532 0.3620 0.3620 0.6661 0.0550 0.0158 0.6127 0.6127 0.7950 0.0569 0.0535 0.0550 0.0550 0.0550 91 table 5 (Cont'd) 9.9131... 012 3 .7 0133 0n ' -7 922 '-7 933 ‘ Univar Bock Fuller Large Sample Fixed Rel. n=50 0.0250 0.0153 1.6838 1.7260 2.2958 011 . .85 022 ‘.85 033 ' .85 Univar Bock Fuller Large Sample Fixed Rel. 0.0185 0.0150 0.1937 0.1938 0.2638 0.0163 0.0166 0.1765 0.1765 0.2613 p11 ' .9 022 3.8 D33 3 .7 Univar Bock Fuller Large Sample Fixed Rel. D1131.“ Univar Bock Fuller Large Sample Fixed Rel. 0.0197 0.0065 0.3682 0.3687 0.6815 0.0173 0.0166 0.0176 0.0176 0.0176 0.0166 0.0035 0.3051 0.3052 0.6308 0.0160 0.0169 0.0160 0.0160 0.0160 .6 023 3 .1 n=100 n=250 .7 0.0207 0.0176 0.0156 0.0152 1.5858 1.6702 1.5911 1.6703 2.1665 2.0319 0.0160 0.0152 0.1706 0.1706 0.2365 0.0161 0.0030 0.2937 0.2937 0.6179 0.0156 0.0150 0.0155 0.0155 0.0155 n=500 0.0165 0.0150 1.6259 1.6259 1.9802 0.0157 0.0153 0.1688 0.1688 0.2350 0.0155 0.0028 0.2861 0.2861 0.6089 0.0155 0.0153 0.0155 0.0155 0.0156 92 Table 5 (Cont'd) m 012'.7 013. .6 9233.6 011 . .7 022 '.7 033 ' .7 Univar Bock Fuller Large Sample Fixed Rel. n850 n8100 0.0118 0.0078 0.0085 0.0069 1.1106 0.8362 1.1260 0.8351 1.5552 1.2132 011 3 .85 022 3.85 D33 3 .05 Univar Bock Fuller Large Sample Fixed Rel. 0.0085 0.0075 0.0952 0.0952 0.1320 0.0068 0.0068 0.0778 0.0778 0.1101 Du 3 .9 D22 3.8 D33 3 .7 Univar Bock Fuller Large Sample Fixed Rel. 01131.00 Univar Bock Fuller Large Sample Fixed Rel. 0.0100 0.0017 0.2532 0.2537 0.3696 0.0078 - 0.0076 0.0078 0.0078 0.0078 0.0072 0.0010 0.1960 0.1960 0.2795 022 =1.oo 033 =1.oo 0.0068 0.0069 0.0068 0.0068 0.0068 n=250 .0065 .0061 .7659 .7659 .1087 #0000 .0059 .0059 .0700 .0700 .1002 00000 .0062 .0008 .1763 .9910 .2569 00000 0.0059 0.0062 0.0060 0.0060 0.0060 .0057 .0057 .0669 .0669 .0961 00000 0.0057 0.0007 0.1683 0.9960 0.2667 0.0057 0.0059 0.0057 0.0057 0.0057 93 94 Summary of counts. The overcorrected correlation counts, the invalid correlation matrices counts, and the counts of the improper variance-covariance matrices of the corrected correlations all show similar patterns. Problems tend to occur when the sample sizes are small, and the reliability triples contain moderate reliability values (e.g., .70). In no case did problems occur when the reliability values were unity. Also, the cases with high percentages for one of the three counts, tended to have high percentages on all three indices. Particularly problematic were the p = (.70, .60, .10) and the p = (.70, .60, .80) cases. When the higher correlations were matched with lower reliabilities, problems were expected, and they did occur. It must be noted, however, that the percentage of times that problems occurred was small for all three indices. No more than 3% of the replications gave out-of-range corrected correlations for any case and reliability combination. The percents were larger for the determinants of the variance- covariances matrices of the corrected correlations, ranging up to 8% with improper values. Finally, the most problematic of the three indices was the determinants of the corrected correlation matrices, showing that up to 15% of the replications in one case yielded improper results. I also recorded the number of times per replication that the Bock and Petersen (1975) and Fuller and Hidiroglou (1978) (iid not default to the univariate correction. The number of 95 cases seems to be associated with the number of invalid correlation matrices. Table 6 displays the cases where the Bock and Petersen and Fuller and Hidiroglou methods did not match the usual univariate correction, these cases are referred to as "adjusted" cases. Ixithese cases, both of the methods adjusted for the problem of a non-positive definite correlation matrix, and used eigenvalues to yield new matrices. These adjustments did not occur very frequently, especially when sample sizes were large. The table displays the number of times per combination that each method "adjusted". Also in the table is the number of replications on which the Fuller and Hidiroglou method required the added adjustment and in the same replication, the univariate and Gleser-method corrected correlation matrices were non—positive definite. Most of the time, the use of the adjustment was related to the invalid nature of the univariate correlation matrix. The Fuller and Hidiroglou correction obviously worked to correct this problem because the method did not yield any problematic correlation matrices. Table 6 Rgghgr 9! Ti!!! 2:: Qggg gherg thg Fuller and Higirgglou and k r en ' tment r needed.‘ Fuller Fuller 6 Case Reliability n Adjust invalid Univ. Corr. Matrix (0, 0, 0) (.7, .7, .7)) 50 0 0 (.6, .3. .7) (.7, .7, .7) 50 36 29 100 0 0 (.9, .8, .7) 50 6 6 (.6, .6, .2) (.7, .7, .7) 50 10 8 (.85, .85, .85) 50 1 0 (.9, .8, .7) 50 O 0 (.6, 6, -.2) ( 7, .7, .7) 50 170 132 100 33 29 (.85, .85, .85) 50 3 3 (.9, .8, .7) 50 12 10 (.7, .6, .1) (.7, .7, .7) 50 363 281 100 93 73 250 6 6 (.85, .85, .85) 50 66 33 (.9, .8, .7) 50 68 38 100 2 1 (.7, .6, .8) (.7, .7, .7) 50 178 152 100 27 19 (.85, .85, .85) 50 3 1 (.9, .8, .7) 50 63 65 100 2 1 Fuller 6 invalid Gleser Corr. Matrix 0 29 132 28 10 281 33 39 169 19 67 ' Only cases where either the Fuller 8 Hidiroglou or Bock 8 Petersen adjustments were needed are included in this table. Bock Adjust 19 ~1qu 162 15 11 19 96 19 Bock & invalid Bock Corr. Matrix 1 16 17 92 19 97 The Book and Petersen method also showed similar results. However, even after the adjustment was applied, many of the resulting corrected correlation matrices were still invalid. An examination of Table 6 along with the raw numbers produced from Table 3 show that virtually any time there was a problem with the corrected correlation matrix, the Fuller and Hidiroglou and Bock and Petersen methods adjusted. These methods also adjusted at other times, but the univariate corrected. correlation. matrices ‘were not necessarily' non- positive definite. These results could be due to rounding error. Results of Magnitude Data Corrected correlations. The average corrected correlations across replications appear in Table 7. These values reflect differences in the methods, as well as how the corrections become more accurate depending on sample size and reliability' values. As shown in the tables, the Bock correction is most different from the others. In fact, when the Bock correction is used in combination with the (.90, .80, .70) reliability, the corrected correlations become much larger than their corresponding population values. Other results visible in the table show that when the reliability values are unity, the univariate, Fuller & Hidiroglou, and Gleser methods all yield identical corrected correlations. Only the Bock and Petersen correction differs. Table 7 "I" rr 2 m. 012 3 .0 013 3 .0 023 3 .0 Pop. Corr. 0.00 0.00 911 ' -7 922 3-7 933 ‘ -7 r e a ions 0000 W 000° -0. '0. -0. -0. Univariate» 0.003 0.001 Bock 0.002 0.002 Fuller 0.002 -0.001 Gleser 0.003 -0.001 011 3 .85 p22 3.05 D33 3 Univariate -0.003 -0.001 Bock o0.007 0.001 Fuller -0.002 -0.001 Gleser -0.002 -0.001 911 ' -9 922 ‘-3 933 ' -7 Univariate -0.006 -0.002 Bock -0.007 -0.006 Fuller -0.003 -0.003 Gleser -0.003 -0.003 011 3 1.00 022 3 1.00 033 ' Univariate 0.002 -0.002 Bock 0.005 -0.003 Fuller 0.002 -0.002 Gleser 0.002 -0.002 0000 0.00 n-50 .006 .006 .005 .005 .006 .006 .005 .005 005 001 006 006 1.00 .005 .008 .005 .005 0.00 -0.001 0.000 -0.001 -0.001 -0.002 -0.006 -0.002 -0.002 -0.002 -0.006 -0.002 -0.002 0.001 0.006 0.001 0.001 n8100 -0 -0 -0 '0 -0. -0. -0. 000° -0. -0. -0. 0.00 .002 .003 .001 .002 001 .000 001 001 .001 .007 .002 .002 .001 003 001 001 0.00 0.000 -0.003 0.001 0.000 -0.001 0.001 -0.001 '0.001 0.002 0.007 0.002 0.002 -0.006 -0.007 -0.006 '0.006 0.00 .003 .002 .003 .003 0000 -0.003 -0.002 -0.002 -0.003 -0.006 -0.003 -0.003 0.000 -0.002 0.000 0.000 0.00 n=250 0.006 0.006 0.006 0.006 0.001 0.002 0.001 0.001 0.000 -0.002 0.000 0.000 0.001 -0.002 0.001 0.001 0.00 -0.001 ~0.003 -0.001 -0.001 0.001 0.000 0.001 0.001 -0.002 -0.002 -0.002 -0.002 -0.002 -0.002 -0.002 -0.002 0.00 0.000 0.001 0.000 0.000 -0.002 -0.002 -0.002 -0.002 0.002 0.006 0.001 0.001 0.00 n=500 0.002 0.001 0.002 0.002 0.000 0.001 0.000 0.000 0.001 0.001 0.001 0.001 0.001 0.000 0.001 0.001 0.00 -0.001 0.000 -0.001 '0.001 '0.002 -0.002 ~0.001 ~0.001 -0.002 '0.002 -0.002 -0.002 table 7 (Cont'd) mplz ' °‘ 013 ' -3 923 ' '1 Pop. Corr. 0.60 0.30 0.10 0.60 0.30 0.10 0.60 0.30 0.10 0.60 0.30 0.10 911 I .7 022 ..7 p33 8 .7 n850 n=100 n=250 n-500 Univariate 0.390 0.305 0.100 0.398 0.296 0.097 0.399 0.299 0.099 0.601 0.300 0.099 Bock 0.633 0.335 0.120 0.628 0.320 0.115 0.616 0.311 0.102 0.608 0.307 0.103 Fuller 0.387 0.301 0.099 0.396 0.295 0.097 0.399 0.298 0.099 0.601 0.300 0.099 Gleser 0.391 0.305 0.100 0.398 0.296 0.097 0.399 0.299 0.099 0.601 0.300 0.099 011 3 .05 022 3.85 D33 3 .85 Univariate 0.396 0.296 0.097 0.602 0.299 0.106 0.399 0.300 0.100 0.601 0.300 0.101 Bock 0.661 0.336 0.120 0.633 0.326 0.122 0.613 0.311 0.105 0.609 0.307 0.105 Fuller 0.395 0.295 0.097 0.602 0.299 0.106 0.399 0.300 0.100 0.601 0.300 0.101 Gleser 0.397 0.296 0.097 0.602 0.299 0.106 0.399 0.300 0.100 0.601 0.300 0.101 911 3 -9 922 '-5 933 ' -7 Univariate 0.395 0.295 0.103 0.398 0.296 0.101 0.602 0.300 0.102 0.601 0.298 0.101 lock 0.535 0.638 0.236 0.567 0.657 0.236 0.556 0.672 0.235 0.556 0.679 0.226 Fuller 0.396 0.293 0.103 0.397 0.295 0.100 0.601 0.300 0.102 0.601 0.297 0.101 Gleser 0.395 0.295 0.106 0.398 0.296 0.101 0.602 0.300 0.102 0.601 0.298 0.101 011 3 1.00 [322 3 1.00 033 3 1.00 Univariate 0.601 0.296 0.106 0.397 0.298 0.096 0.399 0.299 0.099 0.600 0.300 0.101 Bock 0.669 0.336 0.129 0.627 0.323 0.111 0.615 0.312 0.106 0.607 0.306 0.106 Fuller 0.601 0.296 0.106 0.397 0.297 0.096 0.399 0.299 0.099 0.600 0.300 0.101 Gleser 0.601 0.296 0.106 0.397 0.297 0.096 0.399 0.299 0.099 0.600 0.300 0.101 0.70 0.30 0.70 0.60 n8500 100 0.70 0.60 0.30 n=250 0.30 n=100 0.60 0.70 n=50 0.30 TIDlO 7 (Cont'd) mp}; 3 .6 913 3 .3 023 3 .7 mm 011 ' -7 922 '-7 933 ' -7 Pop. Corr. mmmm a e a a 0000 mmmm 0000 ”3”” 363-J 0.0.0.0. 0mm 0.0.0.0. mwmm 0000 can Imlmlfiw. 0.0.0.0. 1 mom 0.0.0.0. mam an 0000 ”5%9 316.366” amnw0.0. 0.627 0.397 0.600 Univariate 0.399 Fuller Gleser Bock 0.301 0.313 0.301 0.301 0.600 0.612 0.399 0.600 0.700 0.726 0.697 0.700 0. 0.318 0.293 0. 0.399 0.601 0.601 0.626 011 3 .05 022 3.05 033 3 .05 011 3 .9 022 3.0 033 3 .7 Univariate lock Fuller Gleser mmmm 0000 mmmm 0000 flm mmmm momm mmmm Univariate Table 7 (Cont'd) E88; ‘3 012 . .6 013 8 0‘ 023 8 .2 Pop. Corr. p11 3 .7 022 8.7 033 3 .7 0.60 0.60 .7 .85 0.200 0.60 n=50 0.199 0.597 0.219 0.615 0.197 0.596 0.199 0.597 0.195 0.602 0.225 0.618 0.196 0.600 0.195 0.601 0.203 0.602 0.375 0.721 0.203 0.601 0.206 0.602 022 . 1.00 033 . 1.00 Univariate 0.598 0.606 Bock 0.628 0.631 Fuller 0.592 0.600 Gleser 0.598 0.606 911 ' ‘5 022 ”“35 933 " Univariate 0.600 0.396 lock 0.632 0.625 Fuller 0.597 0.396 Gleser 0.600 0.396 011 ' -9 022 '-° 033 ' Univariater 0.597 0.398 lock 0.710 0.566 Fuller 0.595 0.397 Gleser 0.597 0.399 011 . 1.00 Univariate 0.601 0.396 Bock 0.635 0.626 Fuller 0.601 0.396 Gleser 0.601 0.396 0.197 0.600 0.223 0.618 0.198 0.600 0.198 0.600 0.60 n=100 0.602 0.619 0.600 0.602 0.397 0.610 0.396 0.397 0.397 0.565 0.396 0.397 0.399 0.616 0.399 0.399 0.20 0.199 0.216 0.198 0.199 0.199 0.210 0.198 0.199 0.200 0.381 0.199 0.200 0.201 0.216 0.201 0.201 0.60 0.60 n=250 0.601 0.606 0.601 0.601 0.600 0.606 0.600 0.601 0.600 0.566 0.600 0.600 0.601 0.609 0.601 0.601 §§§§ 0.60 n=500 0.600 0.606 0.600 0.600 0.600 0.606 0.600 0.600 0.600 0.562 0.600 0.600 0.600 0.602 0.600 0.600 0.20 0.199 0.203 0.199 0.199 0.199 0.202 0.199 0.199 0.203 0.382 0.203 0.203 0.201 0.202 0.201 0.201 102 table 7 (Cont'd) MD” 3 .6 013 ' .4 923 ' '.2 Pop. COff. 0.60 0.40 '0.20 0.60 0.40 ’0.20 0.60 0.40 '0.20 0.60 0.40 '0.20 011 . .7 022 3.7 033 3 .7 ".50 "3100 "3250 "3500 U01VIrilt6 0.597 0.393 '0.195 0.596 0.395 '0.203 0.597 0.402 '0.200 0.599 0.399 '0.201 Bock 0.617 0.400 '0.185 0.612 0.403 '0.106 0.608 0.405 '0.107 0.605 0.402 '0.192 Fuller 0.590 0.307 '0.193 0.593 0.394 ‘0.202 0.596 0.402 “0.200 0.596 0.399 '0.201 Glesar 0.596 0.393 '0.195 0.596 0.396 '0.203 0.597 0.402 '0.200 0.599 0.399 '0.201 911 3 .85 022 3.65 D33 3 .35 Univariate 0.600 0.397 -0.194 0.590 0.403 -0.199 0.600 0.396 -0.204 0.599 0.401 -0.200 Bock 0.619 0.401 -0.174 0.611 0.410 -0.179 0.609 0.400 -0.189 0.605 0.403 -0.193 Fuller 0.598 0.395 -0.194 0.597 0.403 -0.199 0.599 0.396 -0.204 0.599 0.401 -0.200 Gleser 0.600 0.396 -0.195 0.590 0.403 -0.199 0.600 0.396 -0.204 0.599 0.401 -0.200 Du . .9 922 3.. p33 3 .7 Univariate» 0.597 0.399 '0.196 0.599 0.396 '0.199 0.590 0.396 '0.203 0.600 0.399 '0.200 Bock 0.731 0.461 0.016 0.743 0.484 0.032 0.750 0.460 0.037 0.753 0.405 0.043 Fuller 0.595 0.397 '0.194 0.597 0.397 '0.190 0.596 0.396 '0.202 0.600 0.399 '0.200 Gleser 0.597 0.399 '0.195 0.599 0.396 '0.199 0.590 0.396 '0.202 0.600 0.399 ‘0.200 DH 3 1.00 022 3 1.00 033 3 1.00 UniVOFiltO 0.596 0.396 '0.199 0.597 0.390 '0.199 0.600 0.400 '0.190 0.599 0.400 -0.201 Bock 0.616 0.403 '0.173 0.614 0.404 '0.177 0.609 0.404 '0.164 0.605 0.404 ‘0.192 Fuller 0.596 0.396 ‘0.199 0.597 0.390 ‘0.199 0.600 0.400 ‘0.196 0.599 0.400 '0.201 Gleser 0.596 0.396 '0.199 0.597 0.396 '0.199 0.600 0.400 '0.196 0.599 0.400 ‘0.201 103 1.610 7 (Cont'd) Ema-012' -7 013 ' -° 023 ' -‘ Pop. Corr. 0.70 0.60 0.10 0.70 0.60 0.10 0.70 0.60 0.10 0.70 0.60 0.10 p“ a .7 022 8.7 033 8 .7 n-50 M100 n=250 n=500 001VIP1010 0.696 0.599 0.102 0.696 0.600 0.095 0.696 0.590 0.094 0.699 0.599 0.090 060k 0.710 0.619 0.140 0.707 0.611 0.116 0.702 0.604 0.104 0.702 0.602 0.102 F0116? 0.604 0.509 0.101 0.692 0.597 0.095 0.695 0.597 0.094 0.699 0.590 0.090 01686? 0.696 0.599 0.102 0.696 0.600 0.095 0.696 0.590 0.094 0.699 0.599 0.090 011 . .85 022 3.85 p33 3 .85 Univariate 0.699 0.596 0.101 0 699 0.597 0.099 0 700 0.597 0.097 0.699 0.601 0.101 IOCK 0.715 0.613 0.132 0.700 0.600 0.120 0.705 0.602 0.107 0.702 0.603 0.106 F0116? 0.697 0.593 0.100 0.690 0.596 0.099 0.700 0.596 0.097 0.699 0.600 0.101 01680f 0.700 0.595 0.101 0.699 0.597 0.099 0.700 0.597 0.097 0.699 0.601 0.101 011 3 .9 022 3.0 033 3 .7 Univariate» 0.690 0.599 0.100 0.699 0.601 0.104 0.697 0.590 0.097 0.700 0.602 0.103 066k 0.005 0.699 0.353 0.010 0.703 0.366 0.012 0.702 0.360 0.015 0.705 0.370 F0110? 0.694 0.595 0.099 0.690 0.599 0.104 0.697 0.597 0.097 0.700 0.602 0.103 01686? 0.690 0.599 0.100 0.699 0.601 0.105 0.697 0.590 0.097 0.700 0.602 0.103 D“ 3 1.00 022 . 1.00 033 3 1.00 001V6filt£* 0.695 0.599 0.099 0.697 0.597 0.096 0.700 0.600 0.102 0.699 0.599 0.099 006k 0.712 0.619 0.140 0.706 0.600 0.120 0.704 0.604 0.111 0.702 0.602 0.105 F0116? 0.695 0.599 0.099 0.697 0.597 0.096 0.700 0.600 0.102 0.699 0.599 0.099 01686? 0.695 0.599 0.099 0.697 0.597 0.096 0.700 0.600 0.102 0.699 0.599 0.099 104 10610 7 (Cont'd) $113.1. 912 ' -7 p13 ' -° 923 3 -3 0.80 0.70 0.60 0.80 0.70 0.60 0.80 0.70 0.60 0.80 0. 0. Pop. Corr. n=500 n=250 nISO n=100 911 - .7 022 2.7 033 . .7 mmmm o o a a 0000 601 0.602 0.600 0.601 mmmm mm...» nu.0.0.0. 9%%9 5555 nv.0.0.0. mmmm 0000 mmmm 0.0.0.0. ”ms” 5. ...3 0000 OMW1 m. ..w. 0000 mm .mm mmmm o o o o 0000 mmm 0.0.nwnu. C t I i r r... I 00 V01“ .1 l ”“01 UOFG Du ' .05 022 8.05 033 ' .05 Univariate an . .9 022 ..0 033 . .7 0000 mwmw mmmm mmmm 0.0.0.0. 0.0.0.0. mmmm o o o o 0000 mwmm mmmm «(“12 0 0.0.0.0 Univariate 022 :1.00 p33 um 011 - 1.00 mmmm 0.0.0.0. mmmm 0.0.0.0. mmmm 0000 mmmm 0000 ”www O I O 0000 Univariate 105 The Bock and Petersen method yields corrected correlations larger than the other methods. Trends in the table show that higher reliabilities lead to more accurate corrections. Also, the larger the sample size, the more accurate the correction. Overall, the corrected correlations are all remarkably close to the population values, with the exception of the Bock and Petersen correction when the reliabilities are (.90, .80, .70). This may have something to do with reliabilities being unequal in these cases” .An investigation of why these results occurred yielded no solutions. Variances. The estimated variances of the corrected correlations can be compared to the empirical variances based on the sampling distributions (the 2000 cases) of the corrected correlations. Table 8 displays the variances of the corrected. correlations for‘ the different cases and combinations of factors. The third line at the top of each table shows the theoretical variance if one were to substitute the population correlation into the usual variance formula (Equation 3.1). .As shown in the tables, this theoretical variance is smaller than the 'variance of the corrected correlations from the sampling distributions. This result was also found in Becker and Fahrbach (1995). This result was expected, given the work of Bobko and Rieck (1980) (among others) who showed that corrected correlations are more variable than uncorrected correlations. 106 Table 8 Econ Variance: of gorrected Correlations €880 1. p12 . .0 p13 - .0 023 I .0 D D D D 0 D~ 01 D 07 D1 0 9 Correlation 0.0880 0.0880 0.0680 0.0060 0.0580 0.0080 0.00 0 0.0880 0.00 0 0.00 0 0.0880 0.0680 Expect Ver. 0.0200 0.0200 0.0200 0.0100 0.0100 0.0100 0.0040 0.0040 0.0040 0.0020 0.0020 0.0020 Egg. Sam. Var. 0.0445 0.0415 0.0425 0.0213 0.0199 0.0210 0.0083 0.0082 0.0087 0.0041 0.0043 0.0043 Univariate 0.0183 0.0184 0.0184 0.0096 0.0096 0.0096 0.0039 0.0039 0.0039 0.0020 0.0020 0.0020 Large Sample 0.0394 0.0396 0.0396 0.0202 0.0202 0.0201 0.0081 0.0081 0.0081 0.0041 0.0041 0.0041 Fuller 0.0394 0.0396 0.0396 0.0202 0.0202 0.0201 0.0081 0.0081 0.0081 0.0041 0.0041 0.0041 Bock 0.0177 0.0177 0.0176 0.0094 0.0094 0.0094 0.0039 0.0039 0.0039 0.0020 0.0020 0.0020 Fixed Relish. 0.0400 0.0401 0.0402 0.0203 0.0203 0.0203 0.0081 0.0081 0.0081 0.0041 0.0041 0.0041 911‘922’933'-35 Egg. Sam. Var. 0.0297 0.0288 0.0284 0.0143 0.0141 0.0137 0.0056 0.0055 0.0057 0.0027 0.0028 0.0027 Univariate 0.0189 0.0189 0.0189 0.0097 0.0097 0.0097 0.0040 0.0040 0.0040 0.0020 0.0020 0.0020 Large Sample 0.0267 0.0267 0.0268 0.0136 0.0136 0.0136 0.0055 0.0055 0.0055 0.0028 0.0028 0.0028 Fuller 0.0267 0.0267 0.0268 0.0136 0.0136 0.0136 0.0055 0.0055 0.0055 0.0028 0.0028 0.0028 Bock 0.0180 0.0180 0.0181 0.0095 0.0095 0.0095 0.0039 0.0039 0.0039 0.0020 0.0020 0.0020 Fixed Relieb. 0.0269 0.0269 0.0270 0.0136 0.0136 0.0136 0.0055 0.0055 0.0055 0.0028 0.0028 0.0028 p11. .9 0223.8 9333 . 7 Egg. Sam. Var. 0.0283 0.0335 0.0398 0.0150 0.0163 0.0180 0.0057 0.0065 0.0072 0.0027 0 0030 0 0035 Univariate 0.0189 0.0187 0.0185 0.0097 0.0097 0.0097 0.0040 0.0039 0.0039 0.0020 010020 oloozo Large Seaple 0.0269 0.0307 0.0348 0.0138 0.0158 0.0178 0.0055 0.0083 0.0071 0.0028 0.0192 0.0038 Fuller 0.0269 0.0307 0.0348 0.0138 0.0158 0.0178 0.0055 0.0083 0.0071 0.0028 0.0192 0.0038 sock 0.0170 0.0161 0.0170 0.0091 0.0088 0.0092 0.0038 0.0038 0.0038 0.0020 0.0183 0.0020 Fixed Relish. 0.0271 0.0310 0.0350 0.0137 0.0157 0.0177 0.0055 0.0083 0.0071 0.0028 0.0192 0.0038 ”11'922’033" - 0° Egg, Sam. Var, 0.0197 0.0210 0.0208 0.0106 0.0101 0.0101 0.0040 0.0040 0.0044 0.0019 9.0021 0.0019 Univariate 0.0192 0.0192 0.0192 0.0098 0.0098 0.0098 0.0040 0.0040 0.0040 0.0020 0.0020 .00 0 Large Sllple 0.0192 0.0192 0.0192 0.0098 0.0098 0.0098 0.0040 0.0040 0.0040 0.0020 0.0020 0 00 Fuller 0.0192 0.0192 0.0192 0.0098 0 0.0098 0.0040 0.0040 0.0040 0.0020 0.0020 0.00 Bock 0.0184 0.0183 0.0183 0.0095 0. 0.0095 0.0039 0.0039 0.0039 0.0020 0.0020 0 00 Fixed Relieb. 0.0192 0.0192 0.0192 0.0098 0 0.0098 0.0040 0.0040 0.0040 0.0020 0.0020 0 00 107 Table 8 (Cont'd) case 2. 012 ' .4 013 3 .3 023 3 .1 D D D D D D D D D D D D Correlation 0.4000 0.3000 0.1000 0.4000 0.3000 0.1000 0.4000 0.3000 0.1000 0.4000 0.3000 0.1000 Expect Var. 0.0141 0.0166 0.0196 0.0071 0.0083 0.0098 0.0028 0.0033 0.0039 0.0014 0.0017 0.0020 pix-9228033: .7 MW n=100 n=250 n=500 Egg. Sam. Var. 0.0351 0.0352 0.0420 0.0162 0.0187 0.0197 0.0066 0.0075 0.0076 0.0032 0.0035 0.0041 Univeriete 0.0136 0.0156 0.0181 0.0069 0.0080 0.0094 0.0028 0.0033 0.0039 0.0014 0.0016 0.0019 Large Sample 0.0331 0.0358 0.0393 0.0162 0.0179 0.0198 0.0065 0.0072 0.0080 0.0033 0.0036 0.0040 Fuller 0.0331 0.0358 0.0393 0.0162 0.0179 0.0198 0.0065 0.0072 0.0080 0.0033 0.0036 0.0040 Bock 0.0127 0.0145 0.0175 0.0065 0.0078 0.0093 0.0027 0.0032 0.0038 0.0014 0.0016 0.0019 Fixed Relieb. 0.0351 0.0372 0.0399 0.0172 0.0186 0.0200 0.0069 0.0074 0.0081 0.0035 0.0037 0.0040 911'922‘933'-55 Egg, Sam. Var. 0.0214 0.0251 0.0285 0.0105 0.0122 0.0146 0.0041 0.0048 0.0056 0.0021 0.0023 0.0027 Univariate 0.0137 0.0159 0.0186 0.0070 0.0081 0.0095 0.0028 0.0033 0.0039 0.0014 0.0017 0.0020 Large Sample 0.0206 0.0232 0.0264 0.0103 0.0117 0.0134 0.0041 0.0047 0.0054 0.0021 0.0024 0.0027 Fuller 0.0206 0.0232 0.0264 0.0103 0.0117 0.0134 0.0041 0.0047 0.0054 0.0021 0.0024 0.0027 Bock 0.0124 0.0149 0.0179 0.0065 0.0078 0.0094 0.0027 0.0032 0.0039 0.0014 0.0016 0.0019 Fixed Relilb. 0.0215 0.0238 0.0266 0.0108 0.0120 0.0135 0.0043 0.0048 0.0054 0.0022 0.0024 0.0027 0118.9 0223.3 0333.7 E99. 588. Var. 0.0215 0.0273 0.0363 0.0108 0.0140 0.0178 0.0042 0.0053 0.0069 0.0021 0.0027. 0.0036 _— “_fl Univariate 0.0139 0.0160 0.0183 0.0070 0.0081 0.0095 0.0028 0.0033 0.0039 0.0014 0.0016 0.0019 Large Sample 0.0210 0.0273 0.0343 0.0104 0.0136 0.0173 0.0042 0.0055 0.0070 0.0021 0.0027 0.0035 Fuller 0.0210 0.0273 0.0343 0.0104 0.0136 0.0173 0.0042 0.0055 0.0070 0.0021 0.0027 0.0035 Bock 0.0101 0.0121 0.0159 0.0049 0.0060 0.0083 0.0019 0.0024 0.0035 0.0010 0.0012 0.0018 Fixed Ielieb. 0.0219 0.0281 0.0348 0.0108 0.0140 0.0174 0.0044 0.0056 0.0070 0.0022 0.0028 0.0035 ‘1’11'1"22"“":13"’1 - 00 E59. S88, 99?. 0.9150 0.0166 0.0206 0.0076 0.0084 0.0105 0.0028 0.0032 0.0040 0.0014 9.0019 0.0020 Univariate 0.0138 0.0162 0.0188 0.0070 0.0082 0.0096 0.0028 0.0033 0.0039 0.0014 0.0016 0.0020 Large Sample 0.0138 0.0162 0.0188 0.0070 0.0082 0.0096 0.0028 0.0033 0.0039 0.0014 0.0016 0.0020 Fuller 0.0138 0.0162 0.0188 0.0070 0.0082 0.0096 0.0028 0.0033 0.0039 0.0014 0.0016 0.0020 Bock 0.0123 0.0151 0.0182 0.0066 0.0078 0.0095 0.0027 0.0032 0.0039 0.0014 0.0016 0.0019 Fixed Reliab. 0.0138 0.0162 0.0188 0.0070 0.0082 0.0096 0.0028 0.0033 0.0039 0.0014 0.0016 0.0020 108 Table 8 (Cont'd) C88! 30 012 8 0‘ p13 - .3 023 a .7 D D D P D D D 0 Do 0 D 0 Correlation 0.0000 0.3000 0.7000 0.4000 0.3000 0.7000 0.4000 0.3000 0.70 0 0.4000 0.3000 0.7000 Expect Var. 0.0141 0.0166 0.0052 0.0071 0.0083 0.0026 0.0028 0.0033 0.0010 0.0014 0.0017 0.0005 pn-pZZ-pnaj peso n=100 n=250 n=500 Egg. San. Var, 0.0330 0.0383 0.0215 0.0179 0.0189 0.0099 0.0071 0.0073 0.0039 0.0033 0.0037 0.0018 Univariate 0.0134 0.0156 0.0056 0.0069 0.0080 0.0027 0.0028 0.0033 0.0010 0.0014 0.0016 0.0005 Large Sample 0.0325 0.0356 0.0205 0.0164 0.0180 0.0100 0.0065 0.0072 0.0039 0.0033 0.0036 0.0019 Fuller 0.0325 0.0355 0.0205 0.0164 0.0180 0.0100 0.0065 0.0072 0.0039 0.0033 0.0036 0.0019 Bock 0.0128 0.0150 0.0047 0.0068 0.0079 0.0025 0.0028 0.0032 0.0010 0.0014 0.0016 0.0005 Fixed Reliab. 0.0345 0.0370 0.0244 0.0174 0.0187 0.0121 0.0069 0.0074 0.0047 0.0035 0.0037 0.0024 911‘922‘933‘-35 Egg. Sam. Var. 0.0217 0.0244 0.0109 0.0106 0 0121 0.0049 0.0043 0.0050 0.0020 0 0020 0.0024 0.0009 Univariate 0.0138 0.0159 0.0054 0.0070 0.0081 0.0027 0.0028 0.0033 0.0011 0.0014 0.0016 0.0005 Large sample 0.0207 0.0232 0.0103 0.0103 0.0117 0.0050 0.0042 0.0047 0.0020 0.0021 0.0024 0.0010 Fuller 0.0207 0.0232 0.0103 0.0103 0.0117 0.0050 0.0042 0.0047 0.0020 0.0021 0.0024 0.0010 Bock 0.0131 0.0153 0.0047 0.0067 0.0079 0.0024 0.0028 0.0033 0.0010 0.0014 0.0016 0.0005 Fixed Reliab. 0.0216 0.0238 0.0121 0.0108 0.0120 0.0059 0.0043 0.0048 0.0023 0.0022 0.0024 0.0012 0113.9 9223.8 0333-7 Egg. Sam. Var. 0.0215 0.0291 0.0166 0.0107 0.0139 0.0081 0.0041 0.0059 0.0030 0.0022 0.0028 0.0016 Univariate 0.0138 0.0159 0.0057 0.0070 0.0081 0.0027 0.0028 0.0033 0.0011 0.0014 0.0016 0.0005 Large Sample 0.0208 0.0271 0.0167 0.0104 0.0137 0.0080 0.0042 0.0055 0.0031 0.0021 0.0027 0.0015 Fuller 0.0208 0.0271 0.0167 0.010‘ 0.0137 0.0080 0.0042 0.0055 0.0031 0.0021 0.0027 0.0015 Bock 0.0104 0.0127 0.0031 0.0051 0.0064 0.0013 0.0020 0.0025 0.0005 0.0010 0.0013 0.0002 Fixed Reliab. 0.0217 0.0280 0.0199 0.0109 0.0141 0.0096 0.0043 0.0056 0.0038 0.0022 0.0028 0.0019 911'922’033" '00 E52. San. Var. 0.0150 0.0178 0.0057 0.0072, 0.0083 0.0027 0.0028 0.0033 0.0010 0.0014 0.0016 0.0005 Univariate 0.0140 0.0161 0.0054 0.0070 00082 0.0027 0.0028 0.0033 0.0010 0.0014 0.0017 0.0005 Large s-ple 0.0140 0.0161 0.0054 0.0070 0.0082 0.0027 0.0028 0.0033 0.0010 0.0014 0.0017 0.0005 Fuller 0.0140 0.0161 0.0054 0.0070 0.0082 0.0027 0.0028 0.0033 0.0010 0.0010 0.0017 0.0005 Bock 0.0132 0.0154 0.0046 0.0068 00080 0.0025 0.0028 0.0033 0.0010 0.0014 0.0016 0.0005 Fixed Relilb. 0.0100 0.0161 0.0050 0.0070 0.0082 0.0027 0.0028 0.0033 0.0010 0.0014 0.0017 0.0005 Table 8 (Cont'd) C's. ‘- 012 . .6 013 3 .4 023 ' .2 109 0 p 0 D p 09 D D, 09 01 D D~ Correlation 0.61700 0.41800 0.2000 0.6000 0.410300 0.20 0 0.6000 0.4 00 0.20 0 0.6000 0.410100 0.2000 Expcct Var. 0.0082 0.0141 0.0184 0.0041 0.0071 0.0092 0.0016 0.0028 0.0037 0.0008 0.0014 0.0018 pu‘pzzgpn'.7 [1'50 71.100 n=250 0:500 Egg. Sam. Var. 0.0242 0.0351 0.0374 0.0126 0.0174 0.0200 0.0047 0.0065 0.0082 0.0023 0.0031 0.0040 Univariate 0.0082 0.0137 0.0173 0.0042 0.0069 0.0089 0.0016 0.0028 0.0036 0.0008 0.0014 0.0018 Large Sample 0.0248 0.0331 0.0382 0.0124 0.0164 0.0191 0.0048 0.0065 0.0077 0.0024 0.0033 0.0039 Fuller 0.0248 0.0331 0.0382 0.0124 0.0164 0.0191 0.0048 0.0065 0.0077 0.0024 0.0033 0.0039 806k 0.0073 0.0129 0.0167 0.0039 0.0067 0.0087 0.0016 0.0028 0.0036 0.0008 0.0014 0.0018 Fixed 8311.8. 0.0282 0.0350 0.0391 0.0141 0.0174 0.0195 0.0056 0.0069 0.0078 0.0028 0.0035 0.0039 9113922'033‘-35 Egg. Sam. Var. 0.0145 0.0215 0.0274 0.0070 0.0110 0.0133 0 0027 0.0042 0.0054 0.0013 0.0022 0.0026 Univariate 0.0085 0.0137 0.0175 0.0041 0.0070 0.0099 0.0016 0.0028 0.0037 0.0008 0.0014 0.0018 LIFO. Sample 0.0141 0.0206 0.0251 0.0069 0.0104 0.0127 0.0027 0.0042 0.0051 0.0014 0.0021 0.0026 Fuller 0.0141 0.0206 0.0251 0.0069 0.0104 0.0127 0.0027 0.0042 0.0051 0.0014 0.0021 0.0026 806k 0.0075 0.0128 0.0169 0.0039 0.0068 0.0088 0.0016 0.0028 0.0000 0.0008 0.0014 0.0018 Fixed Relilb. 0.0156 0.0215 0.0255 0.0077 0.0108 0.0129 0.0030 0.0043 0.0052 0.0015 0.0022 0.0026 0113.9 022'.a 0333.7 Egg. Sam. Var. 0.0146 0.0251 0.0341 0.0069 0.0127 0.0176 0.0027 0.0052 0.0070 0.0013 0.0023 0.0034 Univariate 0.0083 0.0137 0.0174 0.0042 0.0070 0.0089 0.0016 0.0028 0.0036 0.0008 0.0014 0.0018 Largo Sample 0.0140 0.0243 0.0328 0.0070 0.0122 0.0167 0.0027 0.0049 0.0067 0.0014 0.0024 0.0034 Fuller 0.0140 0.0243 0.0328 0.0070 0.0122 0.0167 0.0027 0.0049 0.0067 0.0014 0.0024 0.0034 006k 0.0051 0.0098 0.0137 0.0024 0.0049 0.0070 0.0009 0.0020 0.0029 0.0004 0.0010 0.0015 Fixed 80l1lb. 0.0155 0.0255 0.0335 0.0078 0.0129 0.0169 0.0030 0.0051 0.0068 0.0015 0.0026 0.0034 911'022‘933“ -°° Egg. 580. V87. 0.0087 0.0150 0.0190 0.0041 0.0072 0.0089 0.0017 0.0027 0.0038 0.0008 0.0015 0.0019 Univariate 0.0083 0.0140 0.0178 0.0041 0.0070 0.0090 0.0016 0.0028 0.0037 0.0008 0.0014 0.0018 L879. Sample 0.0083 0.0140 0.0178 0.0041 0.0070 0.0090 0.0016 0.0028 0.0037 0.0008 0.0014 0.0018 Fuller 0.0083 0.0140 0.0178 0.0041 0.0070 0.0090 0.0016 0.0028 0.0037 0.0008 0.0014 0.0018 Bock 0.0072 0.0132 0.0172 0.0038 0.0068 0.0089 0.0016 0.0028 0.0036 0.0008 0.0014 0.0018 Fixed 0011.8. 0.0083 0.0140 0.0178 0.0041 0.0070 0.0090 0.0016 0.0028 0.0037 0.0008 0.0014 0.0018 Table 8 (Cont'd) CBS. 5. 012 3 .6 013 t .40 923 8 '.2 110 a 0 p 01 p o o p p D p 0 Correlation 0.6000 0.4000 -0. 0 0.600 0.41000 0.2110 0.60011 0.41300 0.2110 0.6000 0.40300 0200 Expect Ver. 0.0082 0.0141 0.0184 0.0041 0.0071 0.0092 0.0016 0.0028 0.0037 0.0008 0.0014 0.0018 pnspzzapnxj n=50 n=100 n=250 n=500 Egg. Sam. Var. 0.0265 0.0355 0.0409 0.0125 0.0163 0.0195 0.0050 0.0065 0.0076 0.0024 0.0033 0.0039 Univariate 0.0084 0.0135 0.0171 0.0042 0.0069 0.0089 0.0017 0.0028 0.0036 0.0008 0.0014 0.0018 large Sunple 0.0252 0.0328 0.0378 0.0123 0.0163 0.0191 0.0049 0.0065 0.0077 0.0024 0.0033 0.0039 Fuller 0.0250 0.0326 0.0376 0.0123 0.0163 0.0191 0.0049 0.0065 0.0077 0.0024 0.0033 0.0039 Spelt 0.0078 0.0131 0.0169 0.0039 0.0067 0.0089 0.0016 0.0028 0.0037 0.0008 0.0014 0.0018 Fixed lleliab. 0.0285 0.0348 0.0388 0.0140 0.0173 0.0194 0.0056 0.0069 0.0078 0.0018 0.0035 0.0039 911’022'033'-35 Egg. Sam. Var. 0.0147 0.0223 0.0270 0.0072 0.0110 0.0131 0.0027 0.0043 0.0054 0.0014 0 0020 0.0027 Univariate 0.0082 0.0138 0.0175 0.0041 0.0070 0.0090 0.0016 0.0028 0.0037 0.0008 0.0014 0.0018 Large Sample 0.0139 0.0206 0.0251 0.0068 0.0104 0.0128 0.0027 0.0042 0.0051 0.0014 0.0021 0.0026 Fuller 0.0139 0.0206 0.0251 0.0068 0.0104 0.0128 0.0027 0.0042 0.0051 0.0014 0.0021 0.0026 Bock 0.0077 0.0132 0.0173 0.0039 0.0069 0.0090 0.0016 0.0028 0.0037 0.0008 0.0014 0.0018 Fixed llelieb. 0.0154 0.0215 0.0255 0.0076 0.0109 0.0129 0.0030 0.0043 0.0052 0.0015 0.0022 0.0026 011..9 022..8 D33..7 Egg. Sam. Var, 0.0136 0.0258 0.0336 0.0068 0.0127 0.0173 0.0028 0.0049 0.0067 0.0014 0.0023 0.0032 Univariate 0.0082 0.0137 0.0174 0.0041 0.0069 0.0090 0.0016 0.0028 0.0036 0.0008 0.0014 0.0018 lerpe Sample 0.0139 0.0244 0.0332 0.0068 0.0122 0.0167 0.0027 0.0049 0.0067 0.0014 0.0024 0.0034 Fuller 0.0139 0.0244 0.0332 0.0068 00122 0.0167 0.0027 0.0049 0.0067 0.0014 0.0024 0.0034 aoclt 0.0045 0.0112 0.0175 0.0020 0.0057 0.0093 0.0008 0.0023 0.0039 0.0004 0.0012 0.0020 Fixed Relieb. 0.0155 0.0257 0.0339 0.0076 0.0128 0.0170 0.0031 0.0051 0.0068 0.0015 0.0026 0.0034 911‘022’933“ -°° Egg. Sam. Var. 0.0086 0.0151 0.0185 0.0041 0.0071 0.0091 0.0017 0.0030 0.0038 0.0009 0.0015 0.0019 Univeriete 0.0084 0.0139 0.0178 0.0041 0.0071 0.0091 0.0017 0.0028 0.0037 0.0008 0.0014 0.0018 Lerpe Sulple 0.0084 0.0139 0.0178 0.0041 0.0071 0.0091 0.0017 0.0028 0.0037 0.0008 0.0014 0.0018 Fuller 0.0084 0.0139 0.0178 0.0041 0.0071 0.0091 0.0017 0.0028 0.0037 0.0008 0.0014 0.0018 Bock 0.0080 0.0135 0.0176 0.0039 0.0069 0.0091 0.0016 0.0028 0.0037 0.0007 0.0014 0.0018 Fixed Ilelieb. 0.0084 0.0139 0.0178 0.0041 0.0071 0.0091 0.0017 0.0028 0.0037 0.0008 0.0014 0.0018 Table 8 (Cont'd) we 6. p12 3 a? p13 3 .6 923 3 .1 Correlation Expect Var. l"11""22"’3:1'°7 Egg. San. Var. Univariate Large Sample Fuller Bock Fixed Relieb. 011'922‘033'-55 EQ. §am, Var. Univariate Large Saeple Fuller Bock Fixed Reliab. 111 0113.9 0223.8 p333.7 Em. Sam. Var. Univariate Large Saeple Fuller Bock Fixed Reliab. 911’922'933" °°° w Univariate Large Seeple Fuller Bock Fixed Reliab. D D 0 p1 DI D 01 D D D D p 0.7000 0.6000 0.1000 0.70 0 0.6000 0.1000 0.7000 0.6000 0.1000 0.7000 0.6000 0.1000 0.0052 0.0082 0.0196 0.0026 0.0041 0.0098 0.0010 0.0016 0.0039 0.0005 0.0008 0.0020 nISO n=100 n=250 n=500 0.0204 0.0251 0.0415 0 0099 0.0123 0 0207 0.0039 0.0048 0.0081 0.0019 0.0022 0.0042 0.0056 0.0082 0.0181 0.0027 0.0042 0.0094 0.0011 0.0016 0.0039 0.0005 0.0008 0.0019 0.0207 0.0247 0.0393 0.0101 0.0123 0.0199 0.0039 0.0048 0.0080 0.0019 0.0024 0.0040 0.0204 0.0244 0.0387 0.0101 0.0123 0.0199 0.0039 0.0048 0.0080 0.0019 0.0024 0.0040 0.0051 0.0076 0.0175 0.0026 0.0040 0.0093 0.0010 0.0016 0.0038 0.0005 0.0008 0.0019 0.0246 0.0281 0.0399 0.0121 0.0140 0.0201 0.0048 0.0056 0.0081 0.0024 0.0028 0.0040 0.0100 0 0134 0 0283 0 0049 o 0072 0 0132 0.0019 0.0027 0.0052 0 0009 0.0013 0.0027 0.0054 0.0084 0.0186 0.0026 0.0041 0.0096 0.0011 0.0016 0.0039 0.0005 0.0008 0.0020 0.0101 0.0141 0.0264 0.0049 0.0069 0.0134 0.0020 0.0027 0.0054 0.0010 0.0014 0.0027 0.0101 0.0141 0.0264 0.0049 0.0069 0.0134 0.0020 0.0027 0.0054 0.0010 0.0014 0.0027 0.0049 0.0079 0.0181 0.0025 0.0040 0.0093 0.0010 0.0016 0.0038 0.0005 0.0008 0.0019 0.0119 0.0156 0.0266 0.0058 0.0077 0.0135 0.0023 0.0030 0.0054 0.0012 0.0015 0.0027 0 0101 0.0172 0.0368 0.0049 0.0086 0.0179 0.0018 0.0033 0.0070 0 0010 0.0016 0.0036 0.0056 0.0084 0.0183 0.0026 0.0041 0.0095 0.0011 0.0016 0.0039 0.0005 0.0008 0.0019 0.0104 0.0174 0.0343 0.0050 0.0084 0.0173 0.0020 0.0034 0.0070 0.0010 0.0017 0.0035 mmm mmn mun 00w000w400n3 0mm mmu mmm 00m000m70065 0.0028 0.0055 0.0143 0.0013 0.0027 0.0072 0.0005 0.0010 0.0029 0.0002 0.0005 0.0015 0.0122 0.0196 0.0347 0.0059 0.0095 0.0175 0.0023 0.0038 0.0070 0.0012 0.0019 0.0035 9.0056 0.0086 0.0195 0.0029 0.0044 0.0105 0.0010 0.0017 0.0041 0.0005 0.0008 0.0019 0.0055 0.0084 0.0189 0.0027 0.0041 0.0096 0.0010 0.0016 0.0039 0.0005 0.0008 0.0020 0.0055 0.0084 0.0189 0.0027 0.0041 0.0096 0.0010 0.0016 0.0039 0.0005 0.0008 0.0020 0.0055 0.0084 0.0189 0.0027 0.0041 0.0096 0.0010 0.0016 0.0039 0.0005 0.0008 0.0020 0.0050 0.0078 0.0182 0.0025 0.0040 0.0094 0.0010 0.0016 0.0038 0. 05 0.0008 0.0019 0.0055 0.0084 0.0189 0.0027 0.0041 0.0096 0.0010 0.0016 0.0039 0.0005 0.0008 0.0020 Table 8 (Cont'd) C888 7e p12 8 .7 013 . 06 023 ‘ .8 Correlation Expect Var. 911'022=933’-7 Egg. Sam. VgrI Univariate Large Sample Fuller Sock Fixed Reliab. 911'922‘933'-35 Egg. Sam. Var. Univariate Large Sample Fuller Bock Fixed Ieliab. 112 0118.9 0223.8 Ont-.7 Egg. Sam. Var. Univariate Large Sample Fuller Bock Fixed Reliab. 9114023933" -°° Univariate Large Sample Fuller Bock Fixed Reliab. 0 o p p 0 p p p 0 p 0 p 0.7000 0.6000 0.8000 0.7000 0.61000 0.8000 0.7000 0.6000 0.8000 0.7000 0.6000 0.8000 0.0052 0.0082 0.0026 0.0026 0.0041 0.0013 0.0010 0.0016 0.0005 0.0005 0.0008 0.0003 "850 n=100 n=250 "8500 0 0220 0.0258 0.0153 0.0099 0.0128 0.0069 0.0038 0.0048 0 0028 0.0018 0.0024 0.0014 0.0057 0.0082 0.0032 0.0027 0.0041 0.0014 0.0011 0.0017 0.0005 0.0005 0.0008 0.0003 0 0209 0.0251 0.0164 0.0100 0.0123 0.0076 0.0039 0.0049 0.0030 0.0019 0.0024 0.0015 0 0208 0.0249 0.0163 0.0100 0.0122 0.0076 0.0039 0.0049 0.0030 0.0019 0.0024 0.0015 0.0052 0.0079 0.0027 0.0026 0.0041 0.0013 0.0010 0.0016 0.0005 0.0005 0.0008 0.0003 0 0248 0.0284 0.0206 0.0120 0.0140 0.0098 0.0048 0.0056 0.0039 0.0024 0.0028 0.0019 0.0103 0.0145 0.0067 0.0047 0.0070 0.0028 0 0019 0 0027 0 0012 0 0010 0.0014 0.0006 0.0054 0.0082 0.0028 0.0027 0.0041 0.0013 0.0010 0.0016 0.0005 0.0005 0.0008 0.0003 0.0102 0.0138 0.0066 0.0050 0.0069 0.0032 0.0020 0.0027 0.0012 0.0010 0.0014 0.0006 0.0102 0.0138 0.0066 0.0050 0.0069 0.0032 0.0020 0.0027 0.0012 0.0010 0.0014 0.0006 0.0052 0.0080 0.0025 0.0026 0.0041 0.0013 0.0010 0.0016 0.0005 0.0005 0.0008 0.0003 0.0120 0.0154 0.0084 0.0059 0.0077 0.0041 0.0023 0.0030 0.0016 0.0012 0.0015 0.0008 0.0109 0.0179 0.0115 0.0047 0.0081 0.0052 0.0020 0.0034 0 0021 0 0010 0.0016 0.0010 0.0055 0.0083 0.0030 0.0027 0.0041 0.0014 0.0011 0.0016 0.0005 0.0005 0.0008 0.0003 0.0105 0.0173 0.0123 0.0050 0.0085 0.0058 0.0020 0.0033 0.0023 0.0010 0.0017 0.0011 0.0104 0.0173 0.0123 0.0050 0.0085 0.0058 0.0020 0.0033 0.0023 0.0010 0.0017 0.0011 0.0039 0.0058 0.0013 0.0019 0.0029 0.0005 0.0007 0.0011 0.0002 0.0003 0.0006 0.0001 0.0122 0.0194 0.0156 0.0059 0.0096 0.0075 0.0024 0.0038 0.0030 0.0012 0 0019 0.0015 0.0054 0 0082 0.0028 0.0027 0.0042 0 0014 0 0011 0 0017 0 0005 0.0005 0.0008 0 0002 0.0054 0.0084 0.0028 0.0027 0.0041 0.0014 0.0010 0.0016 0.0005 0.0005 0.0008 0.0003 0.0054 0.0084 0.0028 0.0027 0.0041 0.0014 0.0010 0.0016 0.0005 0.0005 0.0008 0.0003 0.0054 0.0084 0.0028 0.0027 0.0041 0.0014 0.0010 0.0016 0.0005 0.0005 0.0008 0.0003 0.0051 0.0082 0.0025 0.0026 0.0041 0.0013 0.0010 0.0016 0.0005 0.0005 0.0008 0.0003 0.0054 0.0084 0.0028 0.0027 0.0042 0.0014 0.0011 0.0016 0.0005 0.0005 0.0008 0.0003 113 The other lines in the table correspond to mean variances found from 'various methods. These methods include the univariate method, which simply takes the corrected correlation and inserts it into Equation 3.1 and 3.2. As expected, the results using this method closely approximate the expected variance, and are much smaller than the variance of the sampling distribution. The method from Bock and Petersen, as mentioned previously, did not yield a variance estimate of its own. Instead, the corrected correlations from this method were inserted into the univariate variance formulas for comparison purposes. The Bock and Petersen results are therefore similar to the univariate results. The large-sample theory variances are those found using the formulas in Appendix A and Appendix B. The Fuller and Hidiroglou (1978) method also made use of this formulation, though with differing results. The Fuller and Hidiroglou correction relies on the smallest eigenvalue from the given variance matrix, and if this value is less than unity, a substitution is made. The large-sample theory method with the Fuller and Hidiroglou variation then adjusted for the use of this eigenvalue“ The large-sample variance and the Fuller and Hidiroglou variation of that method should be identical, unless the original correlation matrix is non-positive definite. As shown in the tables, the large-sample variance and the Fuller and Hidiroglou variation give very close 114 approximations to the variance of the sampling distribution of the corrected correlations. The accuracy of these estimators increases as the sample size increases. With E = 500, for example, the elements of the corrected variance-covariance matrices using the large-sample method (and the Fuller and Hidiroglou ‘variation) are within 0.0002 of the sampling distribution, no matter what the reliability values, or the correlation case. The final line in each grouping shows what the variance would be if one assumed that the sample reliability was a constant. This differs from the large-sample formulation in that the large-sample formulation assumes that reliabilities are variable, and that variability is accounted for in the computation. The results using this formulation are quite close to the large-sample results. IHowever, at smaller sample sizes, they are not nearly as accurate as the other variances. The other factor of note is the reliability value. It appears, given these data, to have no effect on the results. What is apparent, however, is that when the sample reliabilities are approximating a population reliability of 1.00, all variance correctionslgive similar results, and.these results closely resemble the expected variance of the correlations measured without error. This result verifies that the simulation seems to be working as it should. table 9 an ri of rr 0 r etion Canton-4013040230.!) 3135353. {18:33 035311. Cove. 0 .0 Expected 01200000 911'922‘933'" m L._-00205 Univar 0.00003 Large-Sn. 0.00006 Fuller 0.00004 lock 0.00004 Fixed Iel. 0.00005 911‘922'93‘45 EQ- Cov, 0.00055 Univar 0.00003 Large-Sn. 0.00001. Fuller 0.00006 lock 4.00001 Fixed Rel. 0.00006 911", 132-.6 038.7 M 0.00162 Univar 0.00009 LargrSu. 0.00009 Fuller 0.00009 lock 0.00009 Fixed Iel. 0.00009 PuW‘“ m 0.00030 mivar 4.00013 Large-Sn. 4.00013 Fuller 4.00013 lock 4.00015 Fixed lel. 4.00013 115 £66630 £06363 31.66386 £6199 035551. £6633 £13556 MSO M100 "I250 "ISM Q . 00105 9.00102 9 . 00006 0 . 0009} 0 . 00042 4 . 00015 4 . 00026 4.0001] 4 . 00003 4.00010 0.00019 4.00010 0.00001 0.00002 4.00001 0.00001 0.00000 4.00001 0.00000 0.00000 0.00026 4.00016 0.00001 0.00003 4.00002 0.00002 4.00001 4.00001 0.00000 0.00000 0.00026 4.00016 0.00001 0.00003 4.00002 0.00002 -0.00001 4.00001 0.00000 0.00000 0.00006 4.00011 4.00003 0.00002 4.00001 0.00001 4.00001 -0.00001 0.00000 ILMOOO 0.00027 4.00015 0.00001 0.00003 4.00002 0.00002 -0.00001 -0.00001 0.00000 0.000” 4 . 0001; 0.00077 4.00007 0.00027 - 0 . 00001 4 . 00013 4 . 00003 0.00000 9 . 00002 4. 00002 4.00008 0.00006 0.00001 4.00002 4.00009 0.00001 0.00001 0.00000 0.00000 0.000(1) 4.00009 0.00037 0.00001 4.00002 4.00010 0.00001 0.00001 0.00000 0.00000 0.W000 4.00009 0.00006 0.00001 -0.00002 4.00010 0.00001 0.00001 0.00000 0.00000 0.00000 4.00016 4.00001 0.00006 -0.m004 4.00009 O.W000 0.00002 0.00001 0.00000 0.00” 4.00009 0.00004 0.00001 4.00002 4.W010 0.00001 0.0M1 0.W000 0.00m 0.00000 L00091 4. 00063 0.00011 4. 00003 0.00078 4 . 0001‘ 4 . OOOLL 0.00017 0.00008 0.00011 0.00015 4.00006 0.00006 0.00001 4.00002 0.00002 4.00001 4.00002 0.00000 0.00000 0.00019 4.00010 0.00005 0.000% -0.00004 0.00002 -0.00001 4.00003 0.00000 0.00”!) 0.00019 4.00010 0.00005 0.000” 4.00006 0.00002 4.WOO1 -0.00003 0.00000 ILWOOO 0.00027 4.00001 0.00005 0.00002 4.0MZ 0.00001 4.00001 -0.00004 0.00000 0.00000 0.00020 4.“!110 0.1!)005 0.00000 -0.00004 0.00002 -0.00001 4.Mfl3 0.00000 0.00000 4.00060 4.00012 4.00009 4.00022 0.00006 4.00002 9 00012 9.00025 4 00001 4.00001 4.00020 0.00001 4.00002 4.00002 4.00002 4.00001 0.00000 0.000” . 0.00000 4.00020 0.00001 4.00002 -0.00002 4.00002 4.00001 0.0011“ 0.00000 0.00000 0.1100” 4.00020 0.00001 -0.00002 -0.00002 -0.00002 -0.00001 0.00000 0.00000 0.00” 0.00000 4.0002! 0.000“) 4.00002 0.00001 4.00005 -0.00001 0.00001 0.m00 4.00001 0.00001 4.00020 0.00001 4.00002 -0.00002 4.0W02 4.00001 0.00000 0.00000 0.00000 0.00000 0 .0 03001010 0 I 00001 0 . 00000 -0.m001 -0.w001 -0.00001 4.00001 4.00001 0.00000 0.00000 Table 9 (Cont'd) 00002.012'.6Du3 e392: Cove. 0 .0 o .0 Expected 0.110061 01.200563 911'922'933'-7 "'50 Egg‘_§g!‘ 0.00008 9.00788 Un1VIr 0.00060 0.00637 LIf’I‘SII. 0.00060 0.00676 Fuller 0.00068 0.00676 lock 0.00050 0.00660 Fixed lel. 0.00121 0.00736 paw-55 £52. £23, 0.00085 Univer 0.00067 terse-Sen. 0.00062 0. Fuller 0.00062 0. lack 0.00050 0 Fixed Iel. 0.00078 0 0117'9 022... 033'.7 L'LEQL 0.00073 0 Univar 0.00056 0 Large-See. 0.00056 0. Fuller 0.00056 0 lock 0.00107 0 Fixed Iel. 0.00000 911%" -°° Univer 0.00053 0.00661 Lorne-Seek 0.00053 0.00661 Fuller 0.00053 0.00661 lock 0.00056 0.00653 Fixed lel. 0.00053 0.00661 I .1 p. .0 0.01160? 0.00923 0.00627 0.00967 0.00967 0.00639 0.01003 0.00657 0.00792 0.00792 0.00691 0.00017 . ,0.1 D 0. 003 0.00059 0.00020 0.00029 0.00029 0.00030 0.00060 .5332 n-100 .00328 0.00223 0.00330 0.00330 0.00220 0.00367 0.00261 0 .0 0.00323 0.00550 0.00336 0.00500 0.00500 0.00350 0.00525 116 0 .0 0.00072 0.00026 0.00012 0.00013 0.00013 0.00012 0.00029 0.00012 0.00027 0.00025 0.00025 0.00020 0.00063 0.00058 _——_—_—_—_ 0.00231 0.00205 0.00335 0.00339 0.00339 0.00339 0.00356 0.00339 0.00012 0.00012 0.00012 0.00012 0.00019 ~0.00009 0.00017 0.00007 0.00012 0.00012 0.00012 0.00011 0.00012 p 00 p ,D D 001 D to? 01,0 0.110051 01.300120 01600 0. 0.100030 n6250 n=500 0.00128 0.00217 -0 00008 0.00073 0.00092 0.00092 0.00137 0.00006 0.00066 0.00069 0.00139 0.00202 0.00007 0.00070 0.00101 0.00139 0.00202 0.00007 0.00070 0.00101 0.00096 0.00161 0.00006 0.00067 0.00070 0.00150 0.00211 0.00015 0.00075 0.00106 0.00110 0 00161 0.00009 0 00059 0.00090 0.00092 0.00130 0 00006 0.00066 0.00070 0.00111 0.00166 0.00006 0.00056 0.00003 0.00111 0.00166 0.00006 0.00056 0.00003 0.00093 0.00161 0.00006 0.00067 0.00071 0.00116 0.00160 0.00009 0.00050 0. 0.00121 0 00182 0.00007 0.00056 0.00107 0.00092 0.00137 0.00006 0.00067 0.00069 0.00119 0.00202 0.00006 0.00060 0.00101 0.00119 0.00202 0.00006 0.00060 0.00101 0.00109 0.00166 0.00009 0.00056 0.00076 0.00126 0.00207 0.00000 0.00063 0.00106 0.00091 0.00135 9.00012 9.00067 0.00076 0.00093 0.00130 0.00006 0.00067 0.00070 0.00093 0.00130 0.00006 0.00067 0.00070 0.00093 0.00130 0.00006 0.00067 0.00070 0.00096 0.00161 0.00006 0.00067 0.00070 0.00093 0.00130 0.00006 0.00067 0.00070 Ieble 9 (Cont'd) m3-Du'.‘013.030n Cove. Explcted 911‘922'030'-7 ML. _0-01‘3‘ Univ-r 0.00947 Large-Sen. 0.01475 Fuller 0.01475 lock 0.00941 Fixed Rel. 0.01630 011.022.0337 .85 0.01241 Univer 0.00971 Large'Sen. 0.01187 Fuller 0.01187 lock 0.00968 Fixed Rel. 0.01258 011" 922" ”if-7 Egg. Cov. 0.01154 Univar 0.00963 Large-Sank 0.01096 Fuller 0.01096 Bock 0.00833 Fixed Rel. 0.01175 911'922‘933" -°° D pp 0 00 0.111013 01.20037 n-so 0.00143 0.00129 0.00196 0.00196 0.00116 0.00384 0.00187 0.00133 0.00151 0.00151 0.00119 0.00239 0.00092 0.00131 0.00160 0.00160 0.00102 0.00289 I .7 010028,? 00.000113 0100023 0.0051g 0.00821 0.00079 0.00274 0.00487 0.00487 0.00257 0.00680 0.00364 0.00269 0.00345 0.00345 0.00250 0.00437 0.00524 0.00278 0.00487 0.00487 0.00224 0.00641 §!2‘_9999 g,0110§ 0.0014; 9.00285 Univer 0.00991 Large-Sen. 0.00991 Fuller 0.00991 lock 0.00973 Fixed Rel. 0.00991 0.00130 0.00130 0.00130 0.00119 0.00130 0.00269 0.00269 0.00269 0.00249 0.00269 0.00494 0.00757 0.00757 0.00495 0.00831 0.0054g_ 0.00497 0.00603 0.00603 0.00493 0.00637 0.00582 0.00500 0.00566 0.00566 0.00426 0.00604 0.00507 0.00501 0.00501 0.00501 0.00496 0.00501 n-100 0.00067 0.00101 0.00101 0.00063 0.00197 0.00080 0.00068 0.00076 0.00076 0.00063 0.00121 0.00071 0.00066 0.00079 0.00079 0.00053 0.00146 0.00079 0.00066 0.00066 0.00066 0.00064 0.00066 {05:71 9.09220 0.00137 0.00239 0.00239 0.00131 0.00337 0.00173 0.00138 0.00175 0.00175 0.00132 0.00221 0.00225 0.00138 0.00240 0.00240 0.00110 0.00321 9.00157 0.00139 0.00139 0.00139 0.00134 0.00139 117 0106592 9.00331 0.00201 0.00305 0.00305 0.00201 0.00334 0.00266 0.00203 0.00245 0.00245 0.00201 0.00258 0.00227 0.00201 0.00227 0.00227 0.00169 0.00242 0.00200 0.00202 0.00202 0.00202 0.00202 0.00202 0 . 0.0.353 n-250 0.00036 0 .0 0¥00056 0.00120 330031132 0 .0 0.0110123 n-soo 0.00010 0.00023 0.00027 0.00030 0.00030 0.00027 0.00048 0.00024 0.00028 0.00033 0.00033 0.00022 0.00061 0.00029 0.00027 0.00027 0.00027 0.00027 0.00027 0.00056 0.00096 0.00096 0.00055 0.00136 0.00066 0.00056 0.00071 0.00071 0.00055 0.00089 0.00093 0.00056 0.00097 0.00097 0.00043 0.00130 0.00054 0.00056 0.00056 0.00056 0.00055 0.00056 0.00101 0.00153 0.00153 0.00101 0.00167 0.00112 9.00013 0.00101 0.00122 0.00122 0.00101 0.00129 0.00121, 0.00101 0.00115 0.00115 0.00085 0.00122 0.00104 0.00102 0.00102 0.00102 0.00101 0.00102 0.00013 0.00020 0.00020 0.00013 0.00040 0.00014 0.00015 0.00015 0.00014 0.00024 0.00020 0.00014 0.00017 0.00017 0.00011 0.00030 0.00017 I 0.00014 0.00014 0.00014 0.00014 0.00014 lel. 9 (Cmt'd) 118 ”O‘cpu'-6Du'-‘pa'.2 CM. 9900! D ,D D 60 D .0 D .0 91.0 05.01 05.099 51.975 019.91 Expected 0. 0006 01.200227 0.0092? 0.110063 0.011219 0.110662 0.00017 0.110005 01.00105 0:000:39 ‘01'pzf¢00'-7 "~50 n-100 n-ZSO Egg, 90VI 0.000102 0.90549 0.01454 10.00004 0.00386 _‘99721 -0.00002 0.99155 0.00274 0.00022 Univar 0.00079 0.00392 0.00071 0.00066 0.00209 0.00639 0.00017 0.00005 0.00102 0.00009 Largo-sun. 0.00084 0.00655 0.01369 0.00050 0.00360 0.00668 0.00010 0.00137 0.00274 0.00009 Fuller 0.00084 0.00655 0.01348 0.00050 0.00360 0.00668 0.00010 0.00137 0.00274 0.00009 lock 0.00075 0.00305 0.00066 0.00041 0.00206 0.00660 0.00017 0.00084 0.00102 0.00000 Fixed Rel. 0.00226 0.00793 0.01465 0.00125 0.00611 0.00726 0.00069 0.00166 0.00296 0.00025 911'922'930'-‘5 £52. Cov. 0.00016 0.00549 0.01656 0.00006 9.00386 0.00721-0.00002 0.00155 0.00276 0.00022 01mm 0.00000 0.00616 0.00063 0.00062 0.00209 0.00650 0.00017 0.00006 0.00184 0.00009 Largo-5.. 0.00076 0.00522 0.01052 0.00039 0.00260 0.00566 0.00015 0.00105 0.00221 0.00000 701101- 0.00074 0.00522 0.01052 0.00039 0.00260 0.00566 0.00015 0.00105 0.00221 0.00000 lock 0.00077 0.00601 0.00059 0.00061 0.00206 0.00650 0.00016 0.0003 0.00184 0.00000 Find :01. 0.00161 0.00500 0.01106 0.00076 0.00294 0.00570 0.00029 0.00110 0.00231 0.00015 011" 922”“ 933'-7 E52. 90VI 0.00049 0.00576 0,01358 9.00055 0.00296 0.00726 0.00017 0.00102 0.00319 9.00005 unmr 0.00072 0.00405 0.00072 0.00062 0.00200 0.00666 0.00017 0.00006 0.00103 0.00009 Largo-s... 0.00065 0.00557 0.01337 0.00039 0.00202 0.00677 0.00016 0.00116 0.00275 0.00000 70116:- 0.00065 0.00557 0.01337 0.00039 0.00202 0.00677 0.00016 0.00116 0.00275 0.00000 lock 0.00107 0.00349 0.00766 0.00050 0.00175 0.00302 0.00026 0.00069 0.00150 0.00012 Fixed 061. 0.00110 0.00627 0.01613 0.00067 0.00317 0.00714 0.00027 0.00120 0.00290 0.00014 911‘9a‘930"-°° E!Q‘_£9!‘ 0.00117 9.00443 0.01000 9.00024 0.00193 0.00455 9.00017 .00094 0.00187 0.00011 011m:- 0.00070 0.00607 0.00096 0.00062 0.00210 0.00656 0.00017 0.00006 0.00106 0.00009 Lam-sun. 0.00070 0.00407 0.00894 0.00062 0.00210 0.00656 0.00017 0.00084 0.00106 0.00009 FUUII’ 0.00070 0.00407 0.00096 0.00062 0.00210 0.00656 0.00017 0.00084 0.00184 0.00009 lock 0.00071 0.00309 0.00896 0.00061 0.00205 0.00653 0.00017 0.00003 0.00166 0.00009 710601101. 0.00070 0.00407 0.00096 0.00062 0.00210 0.00656 0.00017 0.00084 0.00106 0.00009 01 OD 0.0008 "3500 0.00070 0.00043 0.00069 .0.00069 0.00042 0.00083 0.00070 0.00042 0.00053 0.00053 0.00042 0.00059 0.00051 0.00042 0.00057 0.00057 0.00034 0.00064 0.00044 0.00043 0.00043 0.00043 0.00042 0.00043 01 1 l 0? 0.00092 0.00133 0.00092 0.00138 0.00138 0.00092 0.00148 0.00133 0.00092 0.00111 0.00111 0.00092 0.00116 9.00140 0.00092 0.00138 0.00138 0.00081 0.00145 0.00095 0.00092 0.00092 0.00092 0.00092 0.00092 18b“ 9 (Cmt'd) 08885. Du366013.6‘pa"02 119 0 .0. 0.0010 0.00156 0.00156 0.00100 0.00160 9.00126 0.00099 0.00123 0.00123 0.00100 0.00125 9.00143 0.00099 0.00156 0.m156 0.00113 0.W155 9.00100 0.00099 0.0M9 0.00099 0.00100 CM. 0 .D D .D D .D D .D D .D D .D p ,0 D .0 01.0 D .0 D .0 Enacted 01.200298 3.06133 0.01199? 0500119 0.6626? 6.115661 656026 12.63107 0300199 0.61030 6.30051 011-022-033- .7 (ISO 71-100 111-250 MSOO gggfi_9999 -0.00767 0.00987 0.01579_ -0.00349 0.00505 0.00759 -0.00146 0 00198 0.00299 -0.00068 0 00097 thin!” '0.00279 0.00498 0.00908 -0.00146 0.00260 0.00474 '0.00059 0.00106 0.00195 00.00030 0.00053 Large-Sal. °0.00622 0.00931 0.01487 -0.00315 0.00473 0.00760 o0.00127 0.00191 0.00309 -0.00063 0.00095 Fuller ~0.00619 0.00926 0.01477 -o.00314 0.00473 0.00759 -0.00127 0.00191 0.00309 -0.00063 0.00095 lock -0.00271 0.00487 0.00918 -0.00137 0.00256 0.00482 -0.00056 0.00105 0.00198 -0 00029 0.00052 fix-d Iol. ~0.00604 0.00993 0.01555 -0.00301 0.00500 0.00786 -0.00121 0.00202 0.00318 -0.00060 0.00100 paw-IS figgi_9999 -0.00534 0.00741 0.01289 ~0.00267 9.00339 0.00636 -0.00099 0.00147 0.00259_ ~0.00043 0.00076 “fiver ~0.00298 0.00509 0.00947 -0.00149 0.00259 0.00487 -0.00059 0.00105 0.00198 -0.00030 0.00053 Largo-Sal. -0.00450 0.00694 0.01189 '0.00224 0.00350 0.00609 -0.00089 0.00141 0.00246 ~0.00045 0.00071 Fuller -0.00450 0.00694 0.01189 ~0.00224 0.00350 0.00609 -0.00089 0.00141 0.00246 -0.00045 0.00071 lock -0.00268 0.00498 0.00939 -0.00138 0.00253 0.00492 -0.00056 0.00101 0.00200 -0.00029 0.00053 Fixed Rel. -0.00444 0.00724 0.01220 -0.00220 0.00365 0.00623 -0.00087 0.00147 0.00251 -0.00044 0.00074 011.69 0&368 M07 ' £999_9999 -0.00456 0.00703 0.01646 -0 00215 0.00322. 0.00851 ~0.000§§_ 0.00161 0.00314 ~0.00043 0.00080 Univar -0.00292 0.00506 0.00935 -0.00146 0.00259 0.00484 -0.00060 0.00106 0.00196 -0.00030 0.00053 Large-San. -0.00387 0.00765 0.01516 -0.00194 0.00388 0.00774 -0.00079 0.00156 0.00311 -0.00039 0.00078 Fuller -0.00387 0.00765 0.01516 -0.00194 0.00388 0.00774 -0.00079 0.00156 0.00311 -0.00039 0.00078 lock -0.00097 0.00414 0.00997 ~0.00382 0.00205 0.00533 -0.00013 0.00082 0.00223 -0.00006 0.00041 71008 Iol. -0.00426 0.00780 0.01525 '0.00214 0.00393 0.00774 -0.00087 0.00158 0.00309 -0.00043 0.00079 p11""22""‘.10"-"° Egnfi_99!9 -0.00359 9.00512 0.01024 -0.001§§_ 0.00249_ 0.00499 ~0.00067 0.00101 9.00212 ~0.0003z 9.00057 Univar -0.00306 0.00524 0.00955 -0.00151 0.00261 0.00491 -0.00060 0.00107 0.00197 -0.00030 0.00053 LOfflt‘SllL -0.00306 0.00524 0.00955 -0.00151 0.00261 0.00491 -0.00060 0.00107 0.00197 ~0.00030 0.00053 Fuller -0.00306 0.00524 0.00955 -0.00151 0.00261 0.00491 ~0.00060 0.00107 0.00197 -0.00030 0.00053 lock ~0.00278 0.00504 0.00953 -0.00136 0.00256 0.00497 -0.00056 0.00106 0.00199 -0.00029 0.00053 F1008 Iol. -0.00306 0.00524 0.00955 -0.00151 0.00261 0.00491 -0.00060 0.00107 0.00197 -0.00030 0.00053 Table 9 (Cont'd) c... 66 Du . .7 pm I .6 023 I .1 Covs. .0 D .0 D .D Expocud 0211005 05050 0.0007?- L53-0923033-.7 n-50 §!g9_§9!9 -0.00307 9.01063 0.01477 Univar -0.00026 0.00563 0.00605 LIPIO‘SII. -0.00231 0.01052 0.01376 Full-r -0.00226 0.01061 0.01361 lock -0.00029 0.00566 0.00766 F1166 lcl. -0.00092 0.01252 0.01562 "11"".rz“’:u'-as Egg. 99!, -0.00091 0.00861 0.01095 Univar -0.00066 0.00563 0.00667 Largo-$660 -0.00169 0.00756 0.01066 Fuller -0.00169 0.00756 0.01066 lock -0.00032 0.00560 0.00621 Fixed lol. -0.00091 0.00652 0.01171 "11"9 922" 933'3 £999_9999 ~9.00029 9.00928 9.01636 Univar -0.00067 0.00569 0.00625 Large-Sal. '0.00115 0.00666 0.01606 Fuller ~0.00115 0.00663 0.01602 lock 0.00027 0.00369 0.00656 $11.6 Iol. ~0.00102 0.00923 0.01666 911‘922'933"°°° §!p9_gg!b -0. 00M 2 9. 00619 9.008% Univar -0.W 0. 00561 0. 00652 Largt‘6ul0 -0. 00066 0. 00561 0.00652 Fuller -0.00066 0.00561 0.00652 lock -0.00032 0.00550 0.00616 Fix-d Rel. -0.00066 0.00561 0.00652 p l 0072” 013000140111001 -9.00164 -0.00016 ‘0.00116 -0.00116 '0.00016 -0.00066 -0.00096 -0.00016 °0.00069 -0.00069 -0.00015 '0.00039 M151. '0.00016 '0.00051 '0.00051 0.00021 '0.00066 -9.00017 '0.00019 -0.00019 '0.00019 '0.00015 '0.00019 n-100 9. 005% 0 00732 0.00266 0.00527 0.00526 0.00261 0.00625 120 D .0 010087 0. 006B 0. 00705 0.00706 0.00617 0.00792 9 003“ 9 00569 0. 00266 0.00363 0.00363 0.00279 0.00631 0.004 0.00266 0.00625 0.00625 0.00167 0.00666 EEEEE: 0.0073 Lemma; 0.00292 0.00292 0.00292 0.00266 0.00292 0.00631 0.00631 0.00631 0.00620 0.00631 0506082" '0.009§9 9.00229 -0.00006 '0.00066 '0.00066 -0.00006 -0.00016 0.00039 9.00168 '0.00006 '0.00027 -0.00027 -0.00006 '0.00015 '0-000 M °0.00006 ~0.00019 -0.00019 0.00010 -0.00017 -0.00006 -0.00006 -0.00006 -0.00006 -0.00006 '0.00006 .0 0.01 n-250 0.00117 0.00212 0.00212 0.00116 0.00251 0.00117 0.00156 0.00156 0.00116 0.00175 0.00117 0.00171 0.00171 0.00076 0.00167 01%.00310 0.0215. 0.00172 0.00263 0.00263 0.00170 0.00316 0.00316 0.00176 0.00219 0.00219 0.00172 0.00236 0.00299 0.00172 0.00265 0.00265 0.00136 0.00301 0.00119 0.00166 0.00117 0.00117 0.00117 0.00116 0.00117 0.00176 0.00176 0.00176 0.00172 0.00176 .5033? -0.00027 '0.00003 -0.00023 '0.00023 °0.00003 -0.00006 -0.00013 °0.00003 -0.00013 ~0.00013 '0.00003 '0.00007 - .00009 °0.00003 -0.00010 -0.00010 0.00005 '0.00006 -0.00006 '0.00003 -0.00003 '0.00003 -0.00003 -0.00003 0 .0 0. 00 nl500 0.00109 0.00059 0.00105 0.00105 0.00056 0.00125 0.00073 0.00059 0.00076 0.00076 0.00056 0.00067 0.00134 0.00067 0.00163 0.00163 0.00067 0.00160 0.00115 0.00067 0.00110 0.00110 0.00067 0.00116 0.00089 0.00149 0.00059 0.00066 0.00066 0.00036 0.00093 9.00057 0.00059 0.00059 0.00059 0.00059 0.00059 0.00067 0.00163 0.00163 0.00067 0.00151 0.00068 0.00087 0.00067 0.00067 0.00067 0.00067 121 table 9 (Cont'd) m7.pn-.rpu-.6on-.a Cows. .0 . D .D D .D D . D .0 D .D D .D D .D D..D D 2.0 D .0 ...... {M m .02.? .35.. .. ..53 .M. ...w 12...}... .....29 ......‘a . ..a. .30.”... pulping-.7060 M100 n-250 m500 Eggfi_§g!‘ 0.00961 0.00266 0.00596 0.00663 0.00099 0.00250 0.00167 0.00065 9.00100 0.00083 0 00017 0.00065 Univar 0.00666 0.00130 0.00262 0.00226 0.00060 0.00116 0.00090 0.00026 0.00067 0.00065 0.00012 0.00026 Llrflo-Sll. 0.00670 0.00261 0.00529 0.00631 0.00126 0.00256 0.00172 0.00051 0.00102 0.00065 0.00025 0.00051 Fultcr 0.00665 0.00260 0.00526 0.00631 0.00126 0.00256 0.00172 0.00051 0.00102 0.00065 0.00025 0.00051 lock 0.00630 0.00111 0.00227 0.00223 0.00056 0.00113 0.00069 0.00023 0.00066 0.00065 0.00012 0.00023 Fix-d 661. 0.01227 0.00626 0.00690 0.00616 0.00306 0.00666 0.00266 0.00123 0.00176 0.00122 0.00061 0.00069 911'922’933'-55 figQ‘_9995 0.00633 0.00169 0.00360 0.00306 0;00065 0.00156 0.00107 0.00026 0.00060 0.00061 9.00019 0.00030 Univnr 0.00666 0.00119 0.00236 0.00226 0.00059 0.00116 0.00069 0.00026 0.00067 0.00066 0.00012 0.00023 tlruo-Sll. 0.00603 0.00153 0.00333 0.00302 0.00075 0.00165 0.00119 0.00030 0.00066 0.00059 0.00015 0.00033 Fullcr 0.00603 0.00153 0.00333 0.00302 0.00075 0.00165 0.00119 0.00030 0.00066 0.00059 0.00015 0.00033 lock 0.00661 0.00105 0.00222 0.00226 0.00055 0.00116 0.00069 0.00023 0.00067 0.00065 0.00012 0.00023 Fixod lcl. 0.00766 0.00302 0.00696 0.00363 0.00150 0.00267 0.00152 0.00060 0.00099 0.00076 0.00030 0.00069 9111.9 022' .6 an. .7 £529_99__0 .90636 0.00191 9 00699 0. 00267 0.00062 0.00218 0.00115 9.00092 9.00086 0.00056 0.00012 0.00066 Univar 0. 0065 1 0. 00126 0. 00261 0. 00226 0.00060 0.00120 0.00069 0.00026 0.00067 0.00065 0.00012 0.00026 LIPIO'SII. 0. 00566 0. 00166 0. 00530 0. 00270 0.00066 0.00260 0.00107 0.00036 0.00102 0.00056 0.00017 0.00051 Fullor 0.00565 0.00166 0.00530 0.00270 0.00066 0.00260 0.00107 0.00036 0.00102 0.00056 0.00017 0.00051 lock 0.00356 0.00069 0.00135 0.00179 0.00032 0.00066 0.00070 0.00013 0.00025 0.00036 0.00006 0.00012 F1160 Rel. 0.00727 0.00602 0.00607 0.00362 0.00200 0.00602 0.00166 0.00061 0.00160 0.00072 0.00060 0.00060 puma-pgfl .W . 0.00669 0.00117 0.00261A 0.00233 0.00066 0.00130 0.00091 0.00026 0.00069 0 00065 0.00012 9.00023 Univar 0.00651 0.00116 0.00262 0.00226 0.00059 0.00120 0.00069 0.00026 0. 00067 0.00065 0.00012 0.00026 Lorne-Sal. 0.00651 0.00116 0.00262 0.00225 0.00059 0.00120 0.00069 0.00026 0. 00067 0.00065 0.00012 0.00026 Fullcr 0.00651 0.00116 0.00262 0.00225 0.00059 0.00120 0.00069 0.00026 0. 00067 0.00065 0.00012 0.00026 Sock 0.00666 0.00103 0.00225 0.00225 0.00055 0.00115 0.00069 0.00023 0. 00067 0.00065 0.00012 0.00026 Fixod 601. 0.00651 0.00116 0.00262 0.00225 0.00059 0.00120 0.00069 0.00026 0.00067 0.00065 0.00012 0.00026 122 Covariances. The covariances were derived and calculated using the same formulation as the variances, and seem to yield similar results. Table 9 displays the covariances. Again, the univariate and Bock and Petersen corrections produce covariances that are identical or nearly identical to those expected given the population values. And, these two corrections yield covariances which are far different from those found in the empirical sampling distribution of the corrected correlations. The covariance results are not as dramatic as the variance results. Often the sampling distribution values are not close to any of the corrected results. This is especially true in the case (.60, .40, .20), and for the smaller sample sizes (3 = 50 and n = 100) for many of the cases. At the larger sample sizes, again it is clear that the large-sample formulations give results closest to the sampling distribution results, and the univariate corrections seem to give covariances which are too small. An Application of the Methods to Existing Data The methods discussed in this study were used to reanalyze an existing meta—analysis, to see if any differences were apparent, particularly in the decision to accept or reject a homogeneity test calculated from correlation coefficients. This example is a reanalysis of the data from Becker and Cho (1994), which, in turn, was a reanalysis of the. data from Schmidt, Hunter and Outerbridge (1986). A computer 123 program was written in Fortran which allowed the synthesis of the data and the calculation of homogeneity tests. These results were then compared to the results found in Becker and Cho (1994) and Schmidt, Hunter, and Outerbridge (1986). The original Schmidt, Hunter and Outerbridge (1986) study examined four studies, each containing 10 correlations. These ten correlations summarized the relationships among 5 variables: job knowledge, general mental ability, work sample performance, supervisory ratings of job performance, and job experience. Complete data was available for all correlations, and reliability values were given for every measure. Chapter two contains more details about this study. Testing the homogeneity of the correlagion matrices. For this example, the generalized least squares methods used by Becker (1992) will be used. A formal hypothesis test can be used to determine whether the data obtained from several studies are consistent with the hypothesis of a common correlation matrix. Let fl, ..., f4 and rcl, ..., r"4 be the vectors of corrected correlations of length p_(p_+1)/2 = 9* = 10 from each of the 5:4 studies, and let El, ..., 24 be the large-sample covariance matrices of rel, . . ., r°4. The correlations are corrected with the univariate correction. The difference between this example and the methods from Becker and Cho (1994) will be the use of the large-sample theory variance-covariance matrices. Becker and Cho's results used the traditional univariate corrected correlations without 124 adjusting for the variances of the reliabilities and the covariances among the correlations and reliabilities. Define the g p* dimensional vector r, the 15 p_* x 9* matrix X, and the h 9* x 3 p* matrix 2 by r1 I1 r = . , x = . ,and . rk J . Ik J 2 = diag (21, o . o , 2k) , where 11, ..., Ik are identity matrices of order 9* (Becker 1992) . A test of the hypothesis of homogeneity of correlation matrices across studies, that is to test uses the statistic Q = r... [2‘1 - 2‘1 x