Analysis of variance models

Analysis of variance models Abstract: Analysis of variance (ANOVA) models has become widely used tool and plays a fundamental role in much of the application of statistics today. Two-way ANOVA models involving random effects have found widespread application to experimental design in varied fields such as biology, econometrics, quality control, and engineering. The article is comprehensive presentation of methods and techniques for point estimation, interval estimation, estimation of variance components, and hypotheses tests for Two-Way Analysis of Variance with random effects. Key words: Analysis of variance; two-way classification; variance components; random effects model 1. Introduction The random effects model is not fraught with questions about assumptions as is the mixed effects model. Concerns have been expressed over the reasonableness of assuming that the interaction term abij is tossed into the model independently of ai and bj . However, uncorrelatedness, which with normality becomes independence, does seem to emerge from finite sampling models that define the interaction to be a function of the main A and B effects. The problem usually of interest is to estimate the components of variance. The model (1) is referred to as a cross-classification model. A slightly different and equally important model is the nested model. For this latter model see (5) and the related discussion. 2. Estimation of variance components The standard method of moments estimators for a balanced design(i.e., = n ) are based on the expected mean squares for the sums of nij squares. The credentials of the estimators (4) are that they are uniform minimum variance unbiased estimators (UMVUE) under normal theory, and uniform minimum variance quadratic unbiased estimators (UMVQUE) in general. They do, however, suffer the embarrassment of sometimes being negative, except for .e which is always positive. The actual maximum likelihood estimators would occur on a boundary rather than being negative. The best way is to always adjust an estimate to zero rather than report a negative value. It should certainly be possible to construct improved estimators along the lines of the Klotz-Milton-Zacks estimators used in the one-way classification. However, the details on these estimators have not been worked out by anyone for the two-way classification. Estimating variance components from unbalanced data is not as straight-forward as from balanced data. This is so for two reasons. First, several methods of estimation are available (most of which reduce to the analysis of variance method for balanced data), but no one of them has yet been clearly established as superior to the others. Second, all the methods involve relatively cumbersome algebra; discussion of unbalanced data can therefore easily deteriorate into a welter of symbols, a situation we do our best (perhaps not successfully) to minimize here1. On the other hand, extremely unbalanced designs are a horror story. A number of different methods have been proposed for handling them, but all involve extensive algebraic manipulations. The technical detail required to carry out these analyses exceeds the limitations set for this article. On occasion factors A and B are such that it makes no sense to postulate the existence of interactions, so the terms abij should be dropped from (1). In this case .ab disappears from (3) and the estimators for .a and 1 Djordjevic V., Lepojevic V., Henderson?s approach to Variance Components estimation for unbalanced data, Facta Universitatis, Vol.2 No.1, 2004. pg. 59 Another variation on the model (1) gives rise to the nested model. In general, the nested model for components of variance problems occur more frequently in practice than does the cross-classification model. In the nested model the main effects for one factor, say, B, are missing in (1). The reason is that the entities creating the different levels of factor B are not the same for different levels of factor A. For example, the levels (subscript i ) of factor A might represent different litters, and the levels (subscript j) of factor B might be different animals, which are a different set for each litter. The additional subscript k might denote repeated measurements on each animal. To be specific, the formal model for the nested design is: and independence between the different lettered variables. It is customary with this model to use the symbol b rather than ab because the interpretation for this term has changed from synergism or interaction to one of a main effect nested inside another main effect. For a balanced design the method of moments estimators are based on the sums of squares: which have degrees of freedom I-1, I (J-1), and IJ(n-1) , respectively. The mean squares corresponding to (7) have the expectations: The increasing tier phenomenon exhibited in (8) holds for nested designs with more than two effects. The only complication arises when one or more of the estimates are negative. This is an indication that the corresponding variance components are zero or negligible. One might want to resent any negative estimates to zero, combine the adjacent sums of squares, and subtract the combined mean squares from the mean squares higher in the tier. Extension of these ideas to the unbalanced design does not represent as formidable a task for the nested design as it does for the crossed design. The sums of squares (7), appropriately modified for unbalanced designs, form the basis for the analysis. It is even possible to allow for varying numbers Ji of factor B for different levels of factor A. 3. Tests for variance components The appropriate test statistics for various hypothesis of interest can be determined by examining the expected mean squares in the table of analysis of variance. However, we encounter the difficulty that even under the normality assumption exact F tests may not be available for some of the   An analogous F statistic provides a test for H0:.b 2 =0 . Under the alternative no null hypotheses, these ratios are distributed as the appropriate ratios of multiplicative constants from (10) times central F random variables. Thus power calculations are made from central F tables for fixed effects models. The F tests of H :.2 =0 and H :.2 =0 mentioned in the 0 ab 0 a preceding paragraph are uniformly most powerful similar tests. However, they are not likelihood ratio tests, which are more complicated because of boundaries to the parameter space. Although their general use is not recommended because of their extreme sensitivity to no normality, confidence intervals can be constructed based on the distribution theory 10. The complicated method of Bulmer (1957), which is described in Scheffe [11 pg. 27-28], is available. However, the approximate method of Satterhwaite [10 pg. 110-114] may produce just as good results. The distribution theory for the sums of squares (7) used in conjunction with nested designs is straightforward and simple. To test the hypothesis H0:.b2 =0 one uses the F ratio MS (B)/MS(E), and to test H0:.a 2 =0 the appropriate ratio is MS (A)/MS (B). In all nested designs the higher line in the tier is always tested against the next lower line. If a conclusion is reached that .b2 =0 , then the test of H0:.a2 =0 could be improved by combining SS (B) and SS(E) to form a denominator sum of squares with I(J-1) + I J (n-1) degrees of freedom. Under alternative hypotheses these F ratios are distributed as central F ratios multiplied by the appropriate ratio of variances. This can be exploited to produce confidence intervals on some variance ratios. However, one still needs to rely on the approximate Satterhwaite [10 pg. 110-114] approach for constructing intervals on individual components. 4. Estimations of individual effects and overall mean For the two-way crossed classification with random effects interest The classical approach would be to use the estimates ?^ij = yij. The idea would be to shrink the individual estimates toward the common mean as in. where the shrinking factor S depends on the sums of squares SS (E), SS (AB), SS(B), and SS(A) . Unfortunately, the specific details on the construction of an appropriate S have not been worked out for the two-way classification as they have been for the one-way classification. Alternatively, attention might center on estimating a1,, aI , or, equivalently, on the levels of factor B. Again, specific estimators have not been proposed to date for handling this situation. In the nested design one sometimes wants an estimate and confidence interval for ?. One typically uses ?^= y . In the balanced case this estimator has variance. This can be estimated by MS (A)/I J n. In the unbalanced case an estimate for the variability of y can be obtained by substituting estimates .^2, .^b2 and 2 into the expression for the variance of y . Alternative estimators using different weights may be worth considering in the unbalanced case. 5. Conclusion Analysis of variance (ANOVA) models have become widely used tools and play a fundamental role in much of the application of statistics today. In particular, ANOVA models involving random effects have found widespread application to experimental design in a variety of fields requiring Two-Way Analysis of Variance for Random Models measurements of variance, including agriculture, biology, animal breeding, applied genetics, econometrics, quality control, medicine, engineering, and social sciences. With a two-way classification there are two distinct factors affecting the observed responses. Each factor is investigated at a variety of different levels in an experiment, and the combination of the two factors at different levels form a cross-classification. In a two-way classification each factor can be either fixed or random. If both factors are random, the model is called a random effects model. Various estimators of variance components in the two-way crossed classification random effects model with one observation per cell are compared under the standard assumptions of normality and independence of the random effects. Mean squared error is used as the measure of performance. The estimators being compared are: the minimum variance unbiased, the restricted maximum likelihood, and several modifications of the unbiased and the restricted maximum likelihood estimators.