Multivariate GLM, MANOVA, and MANCOVA

Overview Multivariate GLM is the version of the general linear model now often used to implement two long-established statistical procedures - MANOVA and MANCOVA. Multivariate GLM, MANOVA, and MANCOVA all deal with the situation where there is more than one dependent variable and one or more independents. MANCOVA also supports use of continuous control variables as covariates. Multiple analysis of variance (MANOVA) is used to see the main and interaction effects of categorical variables on multiple dependent interval variables. MANOVA uses one or more categorical independents as predictors, like ANOVA, but unlike ANOVA, there is more than one dependent variable. Where ANOVA tests the differences in means of the interval dependent for various categories of the independent(s), MANOVA tests the differences in the centroid (vector) of means of the multiple interval dependents, for various categories of the independent(s). One may also perform planned comparison or post hoc comparisons to see which values of a factor contribute most to the explanation of the dependents. There are multiple potential purposes for MANOVA. To compare groups formed by categorical independent variables on group differences in a set of interval dependent variables. To use lack of difference for a set of dependent variables as a criterion for reducing a set of independent variables to a smaller, more easily modeled number of variables. To identify the independent variables which differentiate a set of dependent variables the most. Multiple analysis of covariance (MANCOVA) is similar to MANOVA, but interval independents may be added as "covariates." These covariates serve as control variables for the independent factors, serving to reduce the error term in the model. Like other control procedures, MANCOVA can be seen as a form of "what if" analysis, asking what would happen if all cases scored equally on the covariates, so that the effect of the factors over and beyond the covariates can be isolated. The discussion of concepts in the ANOVA section also applies, including the discussion of assumptions. See also: Univariate GLM, ANOVA, and ANCOVA Repeated measures GLM Discriminant function analysis, which yields results equivalent to one-way MANOVA.

Contents Key concepts and terms GLM Variables Models Effect size measures Significance tests Post hoc tests Contrast tests Profile analysis / estimated marginal means Canonical correlation Repeated measures Diagnostics Assumptions SPSS output Frequently asked questions Bibliography

Key Concepts

• General Linear Model (GLM). In more recent versions of SPSS, MANOVA and MANCOVA are found under "GLM" (General Linear Model). Output is still similar, but with GLM, parameters (coefficients) are created for every category of every factor and this "full parameterization" approach handles the problem of empty cells better than traditional MANOVA. GLM accepts categorical variables which, in SPSS regression, must be manipulated manually as dummy variables. That is, GLM automatically transforms declared categorical variables into sets of indicator variables. GLM calculates parameters using IWLS (iterative weighted least squares). The seminal article on GLM is Nelder and Wedderburn (1972) and an overview is Gill (2001).
  • GLM vs. MANOVA procedures. Starting with SPSS 7.0, SPSS replaced the MANOVA procedure with the GLM procedure in the menu system. Note, however, that the MANOVA procedure remains available in syntax and has some features not available in GLM, such as correction for violation of sphericity. GLM and MANOVA syntax are listed in the FAQ section below.

  
  

• Variables
  As the initial multivariate GLM dialog in SPSS shows in the figure below, variables are the same as in univariate GLM with the notable exception that there is no support for random factors. Note that the dependents may be nominal (Work Status, wrkstat), ordinal (Happiness, happy), or interval (Number of children, childs). Fixed factors may be nominal (Marital status, marital; Gender, sex) or ordinal (Age category, agecat). Covariates are interval (Years education, educ).
  

  
  

• Models are entered in the same manner as in univariate GLM, in SPSS using the Model button dialog illustrated below. One may specify a full factorial model or custom main and interaction effects.
  

  
  

  • Multiple and multivariate regression models. A multiple regression model in multivariate GLM is simply one with a single continuous dependent variable, no factors, and multiple covariates. A multivariate regression model is the same, except there are multiple continuous dependents. For instance, in the output below, the dependent variables news (low = read newspaper more) and tvhours (high = watch television more) are predicted from respondent's education (educ), father's education (paeduc), and mother's education (maeduc). The SPSS command syntax was "GLM tvhours news WITH maeduc paeduc educ".
    
    The b coefficients cannot be assembled into a regression equation in the same manner as OLS regression due to differences in computation (though predicted values of dependents can be saved to file under the Save button of the SPSS multivariate GLM dialog), but the partial eta² is an R²-like effect size measure for each of the two dependent variables. In the output illustrated above, the "Sig" column indicates that respondent's education (educ) is significantly associated with reading the newspaper more and watching television less, even controlling for parents' education (though parents' education is not significant), explaining a little less than 20% of both media variables.

  • MANOVA and MANCOVA models are the most common uses of multivariate GLM and are the focus of the remainder of this section.

  
  

• Effect size measures
  • Partial eta-square is a nonlinear analog to R-square in regression, similarly interpreted as percent of variance explained. Partial eta² appears in various test tables discussed below when in the SPSS Options button dialog the researcher has checked "Estimates of effect size".

  
  

• Significance

  • Test of overall model significance.
    • F-test. The omnibus or overall F test is the first of the two-step MANOVA process of analysis. The F test appears, illustrated below, in the "Corrected Model" top row of the "Tests of Between-Subjects Effects" table of GLM MANOVA output in SPSS and answers the question, "Is the model significant for each dependent?" There will be an F significance level for each dependent. That is, the F test tests the null hypothesis that there is no difference in the means of each dependent variable for the different groups formed by categories of the independent variables. For the example below, the multivariate GLM is found to be significant for all three dependent variables.
      

  • Test of individual effects overall.
    • Multivariate tests in contrast to the overall F test, answer the question, "Is each effect significant?" or more specifically, "Is each effect significant for at least one of the dependent variables?" That is, where the F test focuses on the dependents, the multivariate tests focus on the independents and their interactions. These tests appear in the "Multivariate Tests" table of SPSS output. The multivariate formula for F is based not only on the sum of squares between and within groups, as in ANOVA, but also on the sum of crossproducts - that is, it takes covariance into account as well as group means. In the figure below, all the main, covariate, and intereaction effects are significant by any of the four leading multivariate tests of group differences, except for the sex*agecat two-way interaction and the single three-way interaction, which are non-significant by all tests but Roy's largest root, which is more liberal.

      Note: To get partial eta-square in this table in SPSS one must check "Estimates of effect size" in the Options button dialog. To get power, one must check "Observed power" also.

      
      • Hotelling's T-Square is the most common, traditional test where there are two groups formed by the independent variables. SPSS prints the related statistic, Hotelling's Trace (a.k.a. Lawley-Hotelling or Hotelling-Lawley Trace). To convert from the Trace coefficient to the T-Square coefficient, multiply the Trace coefficient by (N-g), where N is the sample size across all groups and g is the number of groups. The T-Square result will still have the same F value, degrees of freedom, and significance level as the Trace coefficient. The larger the Hotelling's trace, the more the given effect contributes to the model.

      • Wilks' lambda, U. This is the most common, traditional test where there are more than two groups formed by the independent variables. Wilks' lambda is a multivariate F test, akin to the F test in univariate ANOVA. It is a measure of the difference between groups of the centroid (vector) of means on the independent variables. The smaller the lambda, the greater the differences. The Bartlett's V transformation of lambda is then used to compute the significance of lambda. Wilks's lambda is used, in conjunction with Bartlett's V, as a multivariate significance test of mean differences in MANOVA, for the case of multiple interval dependents and multiple (>2) groups formed by the independent(s). The t-test, Hotelling's T, and the F test are special cases of Wilks's lambda. Wilks' lambda ranges from 0 to 1, and the lower the Wilks' lambda, the more the given effect contributes to the model.

      • Pillai's trace, also called Pillai-Bartlett trace, V. Multiple discriminant analysis (MDA) is the part of MANOVA where canonical roots are calculated. Each significant root is a dimension on which the vector of group means is differentiated. The Pillai-Bartlett trace is the sum of explained variances on the discriminant variates, which are the variables which are computed based on the canonical coefficients for a given root. Olson (1976) found V to be the most robust of the four tests and is sometimes preferred for this reason. The larger the Pillai's trace,.the more the given effect contributes to the model. Pillai's trace is always smaller than Hotelling's trace.

      • Roy's greatest characteristic root (GCR), called "Roy's largest root" in SPSS, is similar to the Pillai-Bartlett trace but is based only on the first (and hence most important) root.Specifically, let lambda be the largest eigenvalue, then GCR = lambda/(1 + lambda). Note that Roy's largest root is sometimes also equated with the largest eigenvalue, as in SPSS's GLM procedure (however, SPSS reports GCR for MANOVA). GCR is less robust than the other tests in the face of violations of the assumption of multivariate normality. The larger the root, the more that effect contributes to the model. Note, however, that Roy's largest root sets a lower bound for the significance value, tending to make this test more prone to Type 1 error (false positives) than the nominal significance level might suggest -that is, it is a more liberal test, as the example above illustrates.

    
    
  • Test of effects on individual dependents.
    • Tests of Between Subjects Effects. As in univariate GLM, the "Tests of Between Subjects Effects," shown in the figure below, applies an F test of significance to the relation of each covariate (educ in the example below), factor (marital, sex, agecat, and each interaction in relation to each of the dependent variables (wrkstat, happy, childs).

      Note: To get partial eta-square in this table in SPSS one must check "Estimates of effect size" in the Options button dialog. To get power, one must check "Observed power" also.

      

      In the Tests of Between Subjects Effects table, partial eta-square serves as an effect size measure. The noncentrality index is used to compute the power level, which by rule of thumb should be equal or greater than .80 to accept with confidence that the chance of Type II error is low enough for a finding of non-significance by the F test (that is, to be confident that the relationship does not exist).

      In the example above, the researcher concludes that the main effect of all factors and covariates is significant except for the effect of sex on happiness, and for that effect there is insufficient power to do more than fail to reject the null hypothesis that sex is unrelated to general happiness. A number of the interaction effects are non-significant in this full factorial model, some with sufficient power (>.80) to conclude no relationship exists (as opposed to merely failing to reject the null hypothesis).

      
      

      • Tests for parameters for factors and covariates
        • Tests of parameter estimates. Parameters(b coefficients) in multivariate GLM are not interpreted as in OLS regression and are not usually a focus of analysis in multivariate GLM. A unit change in an independent variable k does not correspond to a b_k change in the dependent variable as in OLS regression. This is because multivariate GLM uses a nonlinear link function. Parameter estimates in GLM are a necessary intermediate computational step but ordinarily are not reported as they do not lend themselves to the sorts of simple interpretation found for parameter estimates in OLS regression. To compare levels of a factor, one uses contrast analysis, discussed below, not the tests for the b parameters.
          

          The figure above is a partial table. The full table contains similar information for all levels of all effects for each dependent variable. Here only one dependent (happiness) is shown, and not all interaction effect levels are shown even for this one.

          For factors, the univariate t-test significance for levels of a factor (ex., marital in the figure above) refers to the significance of the contrast between the given level and the reference level (the last category if the default is accepted; here, marital = 5 = never married). It is quite possible for these univariate tests to be different from the multivariate test shown in the "Multivariate Tests" and "Tests of Between Subjects Factors" tables above. For instance, the "Marital Status" variable in the parameters table above shows all levels to be non-significant with an effect size of .000, yet the multivariate tests show marital status to be significant overall and for each of the three dependents. Again, the multivariate and between-groups tests, not the b parameters and their t-tests, are the appropriate basis for assessing factors in multivariate GLM and are what are reported.

          For covariates (ex., educ above), the t-test significance level (the "Sig column) will be the same in level as in the F-test in the "Tests of Between Subjects Effects" table above and refers to whether or not the covariate contributes significantly to the model, with the same partial eta² effect size.

  
  

• Multiple comparison and post-hoc tests. Planned comparison and post-hoc tests test the significance of differences in levels of an independent factor with respect to a dependent variable. They are univariate rather than multivariate in nature and thus are applied only after significance is established by multivariate F tests of group differences. The purpose of multiple comparison and post hoc tests are to determine just which group means differ significantly from others. This helps specify the exact nature of the overall effect determined by the F test. Thus if the overall F test shows a given independent variable to be significant, the researcher usually then proceeds to apply planned multiple comparison or post hoc multiple comparison tests as described in the corresponding sections on univariate GLM. Note that covariates must be removed from the model prior to proceeding with multiple comparison and post-hoc tests.
  

  While the reader is referred to the univariate GLM section for discussion of multiple comparison and post-hoc tests, the figure above illustrates two such tests: the Tukey honestly significant difference (HSD) test and the Bonferroni adjusted t-test. Typically the former would be applied for post hoc analysis and the latter for planned comparisons. The table is partial output, just for the "general happiness" dependent variable. Full output covers all dependent variables. While coefficients of significance vary, both tests lead to the same substantive conclusions. For instance, divorced and widowed respondents are not significantly different from never married respondents on general happiness while married and separated respondents are significantly different.

  Since general happiness was coded such that higher scores indicate less happiness, the signs in the mean difference column indicate that never married respondents are significantly less happy (had higher general happiness scores, due to reverse coding) than married respondents while the reverse was true of separated respondents (as indicated by the negative mean difference sign). Asterisks after the mean differences indicate which ones are significant, which is the same as also indicated in the "Sig." column.

  
  

• Contrast Tests are the same as described in the contrasts tests section for univariate GLM, except there will be multiple columns for the multiple dependent variables.

  

  To illustrate briefly, in the figure above, simple contrasts were requested for "Marital status" (marital). Simple contrasts compare the given level with the last level by default. Comparing marital = 1 ("Married) with the reference category (marital = 5 = "Never married") shows a significant difference between the two factor levels (married vs. never married) for general happuiness and number of children, but not for labor force status. In contrast, level 2 ("Widowed") is not significantly different from level 5 on any of the three dependent variables. See the univariate GLM section for further discussion of contrast tests.

  
  

• Profile Analyis
  • Estimated marginal means, as in univariate GLM, show if groups formed by the independent factor (ex., region) vary with respect to a dependent variable (ex., unemployment), controlling for any covariates in the model, as illustrated below.
    

    The more the means of the dependents vary by factor level, the stronger the relation of the factor to the dependent. In the example above, means vary only a little, indicating a weak relationship. By examining the overlap of upper and lower bounds one finds that most regional comparisons are not significant. An exception is South East vs. West for "Having trouble with one's boss," which is significant.

  • Pairwise comparison tables present the same information as the estimated marginal means, but in a different format, illustrated below.
    

    
    

  • Profile plots present the same estimated marginal mean results in graphical form, which is often easier to comprehend (see discussion in the univariate GLM section). Profile plots are line plots of the estimated means of each dependent variable across levels of each factor. When two or three factors are involved, these are called interaction plots. Where a plot of observed means would show the effect being studied and the error, the profile plots of predicted means show the effect without the error. Each point in a profile plot indicates the estimated marginal mean of the dependent variable (adjusted for covariates in MANCOVA) at one level of a given factor. The profile plot shows if the estimated marginal means are increasing across levels.
    

    In profile plots, additional factors can be represented by additional lines, as shown below, where parallel lines indicate no interaction and crossing lines (as in this example) indicate interaction among the factors (here, interaction between education and defaulting on a loan, when dependent variables are types of debt).

    

    
    

  • Profile analysis of within-subjects (repeated) factors is equivalent to repeated measures MANOVA. There is a within-subjects factor, which is either time (the same item administered at sequential time periods) or tests (repeated measures of the same underlying construct). Then there is a between-subjects grouping factor. For instance, tests t1, t2, t3, and t4 could be grouped by gender (m, f). For categories of the grouping factor (here, gender), one could plot the mean response on each of the multiple tests, connecting the means with lines, one line per category of the factor (thus two lines for gender). The lines are termed "profiles." One asks if the profiles are parallel, if the profiles are equal or separated, and if the means of each factor category are the same for each of the dependent variables. In SPSS, profile tests can be accomplished in MANOVA by clicking on the Paste button and adding LMATRIX and MMATRIX commands in syntax to specify the type of contrast wanted, as described below.

    Note that profile analysis in MANOVA has been superceded to some extent by multidimensional scaling, mixed model ANOVA, and/or random effects regression models.

    • Parallelism tests test if profiles are parallel. This is equivalent to testing that there is no interaction of the within-subjects factor with the between-subjects factor (ex., between tests and gender). A finding of non-significance means the profiles are not significantly different in shaper and the researcher concludes the profiles are parallel. In SPSS, the syntax is of the form:

      /LMATRIX = GENDER 1  -1
      /MMATRIX
      t1   1   t2   -1;
      t2   1   t3   -1;
      t3   1   t4   -1. 
      

      The LMATRIX command specifies a contrast between the two values of gender. The MMATRIX command asks for contrasts between t1 and t2, between t2 and t3, and between t3 and t4. Output will be in a section labeled "Custom Hypothesis Tests".

    • Equality of profiles tests, also called separation of group profiles or testing the groups hypotheses. test if parallel profiles are equal (coincident) or separated (dispersed). This is equivalent to testing that there is no main effect of the between-subjects factor (ex., gender). A finding of non-significance means the grouping variable (ex., gender) has no effect and the profiles of gender are equal. In SPSS, the syntax is of the form:

      /LMATRIX = gender   1   -1
      /MMATRIX = t1   .25   t2   .25   t3   .25   t4   .25.
      

      The LMATRIX command again specifies a contrast between the two values of gender. The MMATRIX command asks for a single equality test comparing the four levels of "test" on an equally weighted basis. Output will be in a section labeled "Custom Hypothesis Tests."

    • Equality of means tests, also called tests of flatness, test if the means are equal between factor categories for each dependent (es., between male and female for each of the t1. t2. t3. and t4 tests). This is equivalent to testing if there are significant differences across levels of the within-subjects factor (tests). That is, this is testing if there are significant differences across levels of the within-subjects factor (ex., tests t1, t2, t3, and t4), ignoring the between-groups factor (ex., ignoring gender by looking at the whole sample). No differences would mean a flat profile, hence "tests of flatness." In these tests, the contrast is with the intercept (constant), which represents the equally weighted average of the within-groups factor (the dependent measures, ex. t1, t2, t3, and t4) when the grouping factor is unknown (treated as 0). A finding of non-significance means the gwo genders are not significantly different on the average on any of the levels of the within-groups factor (ex., on t1, t2, t3, and t4). In SPSS, the syntax is of the form:

      /LMATRIX = INTERCEPT   1   gender   .5   .5
      /MMATRIX
      t1   1   t2   -1;
      t2   1   t3   -1;
      t3   1   t4   -1. 
      

      The LMATRIX command specifies a contrast between the two eqaully weighted values of gender and the intercept. The MMATRIX command asks for contrasts between t1 and t2, between t2 and t3, and between t3 and t4. SPSS output will be in a section labeled "Custom Hypothesis Tests."

  
  

• Canonical Correlation
  • In general: The MANOVA procedure but not MANOVA within the multivariate GLM procedure can implement canonical correlation, which is useful for analyzing the relation of a set of covariates to a set of continuous dependents. Categorical factors are not supported on either side of the equation. Implementation is in syntax using MANOVA and is not available in GLM, either through the menu system or in GLM syntax. The meaning of elements of this MANOVA output section is discussed below. See also the separate section on canonical correlation.
    • SPSS example. The brief SPSS example discussed in this section has the following SPSS syntax:

      Input	Data	C:\Program Files (x86)\SPSSInc\Statistics17\Samples\English\survey_sample.sav
      
      Syntax		 
      MANOVA age childs WITH educ paeduc maeduc
      /DISCRIM ALL ALPHA(1)
      /PRINT SIGNIF(MULTIV UNIV EIGEN DIMENR).
      

  • Canonical roots or linear discriminant functions, LDFs. In order to test the hypothesis that groups differ significantly on weighted combinations of the observed independent variables, MANOVA conducts a multiple discriminant analysis (MDA). MDA partitions the variance of dependent variables into components also called canonical roots or LDFs. The canonical roots are analogous to principal components in factor analysis, except they seek to maximize the between-groups variance. In canonical correlation, the canonical roots are also called canonical variables.
    • SPSS example.

      Eigenvalues and Canonical Correlations
      
       Root No.       Eigenvalue           Pct.      Cum. Pct.     Canon Cor.        Sq. Cor
      
              1           .19741       93.10624       93.10624         .40604         .16487
              2           .01462        6.89376      100.00000         .12003         .01441
      

      The output above indicates that the independent set of variables (educ, paeduc, maeduc for respondent's, father's, and mother's education respectively) are related to the dependent set (age, childs, for respondent's age and number of children) by two dimensions. The first dimension explains 93.1% of the variance and the second dimension explains 6.9%. The canonical correlation for the first dimension is .41 and is .12 for the second dimension.

    Each canonical root represents a dimension of meaning, but what? What meaningful label do we give to each canonical root (which SPSS labels merely, 1, 2, etc.)? In factor analysis one ascribes a label to each factor based on the factor loadings of each measured variable on the factor. In MANOVA, this is done on multiple bases, using the standardized weights and structure correlations shown below. The structure correlations are often the most useful for this purpose when there is more than one significant canonical root. Structure correlations are the correlations between the measured variables and the canonical roots. In MANOVA, there will be one set of MDA output for each main and interaction effect.

    • SPSS example.

      - - - - - - - - - - - - - -
       Standardized canonical coefficients for DEPENDENT variables
                 Function No.
       Variable                  1                2
       age                  .79451          -.78299
       childs               .34987          1.05920
      - - - - - - - - - - - - - -
      
       Correlations between DEPENDENT and canonical variables
                 Function No.
       Variable                  1                2
       age                  .94954          -.31365
       childs               .70192           .71225
      - - - - - - - - - - - - - -
      

      The output above shows age loads heavily in a positive direction on dimension 1. Age loads negative and childs loads positively on dimension 2. Dimension 2 is most heavily correlated with childs. Howevever, if the researcher labels dimension 1 as "the age dimension" and dimension 2 as the "nnumber of children dimension," this is a half-truth oversimplification. As in factor analysis, it is difficult to label dimensions based on loadings and correlations.

  • Eigenvalue: The eigenvalue has to do with the proportion of the total variance of the group of variables included in the analysis which is accounted for by a specific canonical root. That is, the larger the eigenvalue, the larger the group differences on the variate (the variable computed by the linear combination of canonical coefficients) for that canonical root. SPSS will print out the associated percent of variance explained and cumulative percent next to each canonical root in the section "Eigenvalues and Canonical Correlations," illustrated above.

  • Canonical correlation: Every main and interaction effect has a set of canonical roots, and every canonical root has both a corresponding eigenvalue and canonical correlation coefficient. The canonical coefficients are the weights in the linear combination of canonical roots relating the independent and dependent sets of variables being canonically correlated, which maximize the correlation relating the two sets of variables. In MANOVA there will be a set of canonical roots for each main and interaction effect of the independents on the set of dependent variables, and there will be one canonical coefficient for each canonical root. Since canonical correlation coefficients are standardized, their weights may be compared. The ratio of canonical correlation weights for a set of canonical roots is their relative importance for the given effect.

    See also the separate section on canonical correlationand the annotated MANOVA output found there.

  
  
• Repeated Measures MANOVA.
  • SPSS. Select Analyze, General Linear Model, Repeated. Repeated measures MANOVA is discussed separately.

  
  

• Diagnostics
  • Lack of fit test. Though not illustrated here, output for multivariate GLM contains lack of fit tests as in univariate GLM, used when the researcher has employed a custom (non-full factorial) model and wishes to test if terms present in the full factorial model but absent in the custom model should have been included. See the lack of fit test output in the section on univariate GLM. It is selected in the Options button dialog of multivariate GLM.

  • Spread-versus-level plots show which factor levels deviate substantially from MANOVA's assumption of homogeneity of variances. They do this by graphing standard deviations vs. means, or variances vs. means, for each dependent variable. Each point shows the value of a factor design matrix group cell on the mean and on the standard deviation or variance. In the example below on the left, the West level of the Region factor exhibits quite different and higher variance than the North East and South East. Moreover, as the group mean on the dependent increases, varkiance decreases linearly. In the plot on the right below there is more of the desirable random cloud pattern except that all degree/work status groups with a mean income above 11 have distinctly low variance. Spread vs. level plots are selected in the Plots button dialog of SPSS multivariate GLM.
    

  • Residual analysis with observed*predicted*standardized residual plots. For each dependent variable, a plot is produced which shows the 6 comparisons among observed, predicted, and standardized residuals. For observed by predicted, one would like to see a clear pattern approaching linearity, but for the plots involving standardized residuals, one would like to see a cloud-like random pattern. The figure below illustrates two such plots, with the one on the left for a dichotomous dependent variable and the one on the right for a continuous dependent. These plots are also selected in the Plots button dialog of SPSS multivariate GLM.
    

    For the credit card case on the right, for observed by predicted there is a mostly random pattern, indicative of a weak or nonexistent relationship between the predictors and the dependent. The upward linear pattern of residuals by observed shows that as the observed value of credit card debt goes up, model error increases. For the example on the left, for a dichotomous dependent, as the observed value goes from 0 to 1, predicted values are a bit higher but the overlap is very large, again indicating a weak or possibly nonsignificant relationship. With a dichotomy, the residual plots cannot assume a random cloud pattern, but it is again seen that as observed increases in value, so does residual (error) value.

    
    

Assumptions

• Observations are independent of one another. The usual MANOVA (in SPSS, Analyze, General Linear Model, Multivariate) is not robust when the selection of one observation depends on selection of one or more earlier ones, as in the case of before-after and other repeated measures designs. This is the most serious MANOVA assumption to violate. However, there does exist a variant of MANOVA for repeated measures designs (select Analyze, General Linear Model, Repeated).

• The independent variable is or variables are categorical.

• The dependent variables are continuous and interval level.

• Low measurement error of the covariates. The covariate variables are continuous and interval level, and are assumed to be measured without error. Imperfect measurement reduces the statistical power of the F test for MANCOVA and for experimental data, there is a conservative bias (increased likelihood of Type II errors: thinking there is no relationship when in fact there is a relationship) . As a rule of thumb, covariates should have a reliability coefficient of .80 or higher.

• Equal group sizes. To the extent that group sizes are very unequal, statistical power diminishes. Conversely, if group sizes are approximately equal, MANOVA is robust against violations of the homoscedasticity assumption discussed below. SPSS takes unequal group sizes into account. In SPSS, METHOD=UNIQUE is the usual method.

• Appropriate sums of squares. Normally there are data for every cell in the design. For instance, 2-way ANOVA with a 3-level factor and a 4-level factor will have 12 cells (gropups). But if there are no data for some of the cells, the ordinary computation of sums of squares ("Type III" is the ordinary, default type) will result in bias. When there are empty cells, one must ask for "Type IV" sums of squares, which compare a given cell with averages of other cells. In SPSS, Analyze, General Linear Model, Univariate; click Model, then set "Sum of Squares" to "Type IV" or other appropriate type depending on one's design:
  1. Type I. Used in hierarchical balanced designs where main effects are specified before first-order interaction effects, and first-order interaction effects are specified before second-order interaction errects, etc. Also used for purely nested models where a first effect is nested within a second effect, the second within a third, etc. And used in polynomial regression models where simple terms are specified before higher-order terms (ex., squared terms).

  2. Type II. Used with purely nested designs which have main factors and no interaction effects, or with any regression model, or for balanced models common in experimental research.

  3. Type III. The default type and by far the most common, for any models mentioned above and any balanced or unbalanced model as long as there are no empty cells in the design.

  4. Type IV. Required if any cells are empty in a balanced or unbalanced design. This would include all nested designs, such as Latin square design.

• Adequate sample size. Small samples may have lower power. At a minimum, every cell must have more cases than there are dependent variables. With multiple factors and multiple dependents, group sizes fall below minimum levels more easily than in ANOVA/ANCOVA. The larger the sample, the more robust MANOVA/MANCOVA is against violation of assumptions.

• Residuals are randomly distributed, forming a multivariate normal distribution around a mean of zero.

• Homoscedasticity (aka "homogeneity of variances and covariances" or "homogeneity of covariance matrices"): within each group formed by the categorical independents, the error variance of each interval dependent should be similar, as tested by Levene's test, below. Also, for each of the k groups formed by the independent variables, the covariance between any two dependent variables must be the same. When sample sizes are unequal, tests of group differences (Wilks, Hotelling, Pillai-Bartlett, GCR) are not robust when this assumption is violated. Pillai-Bartlett trace was found to be more robust than the alternatives when this assumption was violated but sample sizes of the groups were equal (Olson, 1976).
  • Box's M: Box's M tests MANOVA's assumption of homoscedasticity using the F distribution. If p(M)<.05, then the covariances are significantly different. Thus we want M not to be significant, rejecting the null hypothesis that the covariances are not homogeneous. That is, the probability value of this F should be greater than .05 to demonstrate that the assumption of homoscedasticity is upheld. Note, however, that Box's M is extremely sensitive to violations of the assumption of normality, making the Box's M test less useful than might otherwise appear. For this reason, some researchers test at the p=.001 level, especially when sample sizes are unequal. In the example of SPSS output below, Box's M shows the assumption of equality of covariances among the set of dependent variables is violated with respect to the groups formed by the categorical independent factor "region".
    

  • Levene's Test. SPSS also outputs Levene's test as part of Manova. This is discussed in the section on ANOVA. If Levene's test is significant, then the data fail the assumption of equal group error variances. In the figure below, Levene's test shows that the assumption of homogeneity of error variances among the groups of "region" is violated for two of the three dependent variables listed.
    

• Homogeneity of regression. The covariate coefficients (the slopes of the regression lines) are the same for each group formed by the categorical variables and measured on the dependents. The more this assumption is violated, the more conservative MANCOVA becomes (the more likely it is to make Type I errors - accepting a false null hypothesis). When running a MANCOVA model in SPSS, include in the model options the interactions between the covariate(s) and each independent variable -- any significant interaction effects indicate that the assumption of homogeneity of regression coefficients has been violated. See the discussion in the section on testing assumptions.
  • Linearity. As a corollary, within each group formed by the categorical variables, the relationship of each dependent variable to another dependent should be linear as should the relationship of each covariate to each dependent, and each covariate to each other covariate. Violations of this linearity assumption reduce the power of the procedure. This may be checked visually by running scatterplots of each pair of covariates and dependents by group (thus by factor levels if there is one categorical factor).

• Sphericity. In a repeated measures design, the univariate ANOVA tables will not be interpreted properly unless the variance/covariance matrix of the dependent variables is circular in form (see Huynh and Mandeville, 1979). A spherical model implies that the assumptions of multiple univariate ANOVA is met, that the repeated contrasts are uncorrelated. When there is a violation of this assumption in a repeated measures design, the researcher should use MANOVA rather than multiple univariate ANOVA tests.
  
  • Bartlett's test of sphericity

    • Bartlett's test prior to MANOVA. Bartlett's test can be used to determine if the researcher is warranted in going ahead with a multiple analysis of variance rather than a series of univariate analyses. Such analysis would be inappropriate if on the whole, dependent variables were not significantly intercorrelated - which is what Bartlett's test tests. Thus it tests the null hypothesis that the covariance matrix is proportional to an identity matrix (an identity matrix would indicate all dependent variables were uncorrelated). A finding of significance means the null hypothesis is rejected, meaning that at least one of the inter-correlations is non-zero. Thus, if Bartlett's test is significant, the researcher concludes that it is appopriate to proceed with multiple analysis of variance or with factor analysis. To obtain Bartlett's test in GLM univariate, check "Descriptive Statistics" under the Options button.

    • Bartlett's test within MANOVA. When applied to the residual covariance matrix in the context of MANOVA, Bartlett's test is testing if there are still significant correlations among the dependent variables after controlling for variables in the MANOVA model. In a good model, the residual correlations approach zero because the residuals are random. Thus in the context of MANOVA, the researcher wants a finding of non-significance on Bartlett's test in order to uphold the MANOVA model. To obtain Bartlett's test, check "Residual SSCP" under the Options button in the SPSS Multivariate GLM dialog.

    • Bartlett's test in repeated measures MANOVA tests for the assumption of sphericity, required by repeated measures designs, to assure that the variance of effect differences is equal across time groups. Bartlett's test in repeated measures is discussed in the separate section on repeated measures GLM.

    Note that Bartlett's test is most useful for small samples: in large samples Bartlett's test may be too sensitive (too likely to find significant correlations).

  • Mauchly's test of sphericity. This alternative sphericity test is derogated because it lacks power for small samples (it fails too often to identify violations of sphericity) and it is too conservative for large samples (too small violations of sphericity are found significant).

  • Corrections for violation of sphericity. Two main procedures have been used to correct for lack of sphericity in repeated measures designs. Both are available in SPSS only in syntax, using the MANOVA rather than GLM procedure.

    • Huynh-Feldt correction. The epsilon adjustment factor is computed which is used to adjust the degrees of freedom in the F test of sphericity. The adjustment, which is based on the degree of heterogeneity of variances, makes the critical value of F larger, thus making a finding of significance more difficult.

    • Geisser-Greenhouse (lower bound) correction.. In this correction, epsilon is computed differently and then used to correct the degrees of freedom more than the Huynh-Feldt correction. That is, this is a more severe correction which is more conservative (a finding of significance is even more difficult) While GG correction has lower Type I error, its power is low, leading to too many Type II errors. .

• Multivariate normal distribution. For purposes of significance testing, response (dependent) variables are assumed to have multivariate normal distributions. In practice, it is common to assume multivariate normality if each variable considered separately follows a normal distribution. MANOVA is robust in the face of most violations of this assumption if sample size is not small (ex., under 20) and there are no outliers.

• No outliers. MANCOVA is highly sensitive to outliers in the covariates, its robustness against other types of non-normality notwithstanding.

• Covariates are linearly related or in a known relationship to the dependents. The form of the relationship between the covariate and the dependent must be known and most computer programs assume this relationship is linear, adjusting the dependent mean based on linear regression. Scatterplots of the covariate and the dependent for each of the k groups formed by the independents is one way to assess violations of this assumption. Covariates may be transformed (ex., log transform) to establish a linear relationship.

• Does not assume linear relationships; can handle interaction effects.

• See also: assumptions sections for ANOVA and ANCOVA.

  
  

SPSS Output Examples, with Commentary

• MANOVA
• MANCOVA

Frequently Asked Questions

• What is the multivariate GLM syntax in SPSS?

  GLM dependent varlist [BY factor list [WITH covariate list]]
  [/REGWGT=varname]
  [/METHOD=SSTYPE({1  })]
                  {2  }
                  {3**}
                  {4  }
  [/INTERCEPT=[INCLUDE**] [EXCLUDE]]
  [/MISSING=[INCLUDE] [EXCLUDE**]]
  [/CRITERIA=[EPS({1E-8**})] [ALPHA({0.05**})]
                  {a     }          {a     }
  [/PRINT  = [DESCRIPTIVE] [HOMOGENEITY] [PARAMETER][ETASQ] [RSSCP]
             [GEF] [LOF] [OPOWER] [TEST [([SSCP] [LMATRIX] [MMATRIX])]]
  [/PLOT=[SPREADLEVEL] [RESIDUALS]
         [PROFILE (factor factor*factor factor*factor*factor ...) [WITH(covariate={value} [...])]]
                                                                                  {MEAN }
  [/LMATRIX={["label"] effect list effect list ...;...}]
            {["label"] effect list effect list ...    }
            {["label"] ALL list; ALL...               }
            {["label"] ALL list                       }
  [/MMATRIX= {["label"] depvar value depvar value ...;["label"]...}]
             {["label"] depvar value depvar value ...             }
             {["label"] ALL list; ["label"] ...                   }
             {["label"] ALL list                                  }
  [/KMATRIX= {list of numbers    }]
             {list of numbers;...}
  [/SAVE=[tempvar [(list of names)]] [tempvar [(list of names)]]...]
         [DESIGN]
  [/OUTFILE=[{COVB('savfile'|'dataset')}] 
             {CORB('savfile'|'dataset')}
            [EFFECT('savfile'|'dataset')] [DESIGN('savfile'|'dataset')]
  [/DESIGN={[INTERCEPT...]    }]
           {[effect effect...]}
  
  ** Default if the subcommand or keyword is omitted. 
  
  
  Example 
  GLM SCORE1 TO SCORE4 BY METHOD(1,3).
  

• What is the MANOVA syntax in SPSS?
  The MANOVA procedure, different from the GLM procedure, is only available in syntax. While largely superceded by GLM, MANOVA does have some functionality not present in GLM, such as correction for violation of sphericity.

  MANOVA dependent varlist [BY factor list (min,max)[factor list...]
                           [WITH covariate list]]
   [/WSFACTORS=varname (levels) [varname...] ]
   [/WSDESIGN]*
   [/TRANSFORM [(dependent varlist [/dependent varlist])]=
                       [ORTHONORM] [{CONTRAST}]  {DEVIATION (refcat)   } ]
                                    {BASIS   }   {DIFFERENCE           }
                                                 {HELMERT              } 
                                                 {SIMPLE (refcat)      }
                                                 {REPEATED             }
                                                 {POLYNOMIAL [({1,2,3...})]} 
                                                 {             {metric  }  }
                                                 {SPECIAL (matrix)     }
   [/MEASURE=newname newname...]
   [/RENAME={newname} {newname}...]
            {*      } {*      }
   [/ERROR={WITHIN           } ]
           {RESIDUAL         }
           {WITHIN + RESIDUAL}
           {n                }
   [/CONTRAST (factorname)={DEVIATION** [(refcat)]    }] †
                           {POLYNOMIAL**[({1,2,3...})]} 
                           {              {metric  }  }
                           {SIMPLE [(refcat)]         }
                           {DIFFERENCE                }
                           {HELMERT                   }
                           {REPEATED                  }
                           {SPECIAL (matrix)          }
   [/PARTITION (factorname)[=({1,1...  })]] 
                              {n1,n2...}
   [/METHOD=[{UNIQUE**  }]  [{CONSTANT**}] [{QR**    }]]
             {SEQUENTIAL}    {NOCONSTANT}   {CHOLESKY}	
   [/{PRINT  }= [CELLINFO [([MEANS] [SSCP] [COV] [COR] [ALL])]]    
     {NOPRINT}  [HOMOGENEITY [([ALL] [BARTLETT] [COCHRAN] [BOXM])]] 
                [DESIGN [([OVERALL] [ONEWAY] [DECOMP] [BIAS] [SOLUTION]
                         [REDUNDANCY] [COLLINEARITY] [ALL])]]
                [PARAMETERS [([ESTIM] [ORTHO][COR][NEGSUM][EFSIZE][OPTIMAL][ALL])]]
                [SIGNIF [[(SINGLEDF)]
                         [(MULTIV**)] [(EIGEN)] [(DIMENR)]
                         [(UNIV**)] [(HYPOTH)][(STEPDOWN)] [(BRIEF)]
                         [{(AVERF**)}] [(HF)] [(GG)] [(EFSIZE)]]
                          {(AVONLY) }
               [ERROR[(STDDEV)][(COR)][(COV)][(SSCP)]]
   [/OMEANS =[VARIABLES(varlist)] [TABLES ({factor name     }] ]
                                           {factor BY factor}
                                           {CONSTANT        }
   [/PMEANS =[VARIABLES(varlist)] [TABLES ({factor name     })] [PLOT]] ]
                                           {factor BY factor}
                                           {CONSTANT        }
   [/RESIDUALS=[CASEWISE] [PLOT] ]
   [/POWER=[T({.05**})] [F({.05**})] [{APPROXIMATE}]] 
              {a    }      {a    }    {EXACT      }
   [/CINTERVAL=[{INDIVIDUAL}][({.95}) ] 
                {JOINT     }   {a  }    
               [UNIVARIATE ({SCHEFFE})]
                            {BONFER }  
               [MULTIVARIATE  ({ROY      })]  ] 
                               {PILLAI   }
                               {BONFER   }
                               {HOTELLING}
                               {WILKS    }
   [/PCOMPS [COR] [COV] [ROTATE(rottype)]
            [NCOMP(n)] [MINEIGEN(eigencut)] [ALL] ]
   [/PLOT=[BOXPLOTS] [CELLPLOTS] [NORMAL]  [ALL] ] 
   [/DISCRIM [RAW] [STAN] [ESTIM] [COR] [ALL] 
             [ROTATE(rottype)] [ALPHA({.25**})]] 
                                      {a    }
   [/MISSING=[LISTWISE**] [{EXCLUDE**}] ]
                           {INCLUDE  } 
   [/MATRIX=[IN({file})]  [OUT({file})]]
                {[*] }         {[*] }
   [/ANALYSIS [({UNCONDITIONAL**})]=[(]dependent varlist
                {CONDITIONAL    }      [WITH covariate varlist]
                                        [/dependent varlist...][)][WITH varlist] ]
   [/DESIGN={factor [(n)]  }[BY factor[(n)]] [WITHIN factor[(n)]][WITHIN...]
            {[POOL(varlist)}
            [+ {factor [(n)] }...]
               {POOL(varlist)}
            [[= n] {AGAINST} {WITHIN  }
                   {VS     } {RESIDUAL}
                             {WR      }
                             {n       }
            [{factor [(n)] } ... ] 
             {POOL(varlist)}  
            [MWITHIN factor(n)]
            [MUPLUS]
            [CONSTANT [=n] ]
  
  
  * WSDESIGN uses the same specification as DESIGN, with only within-subjects factors.
  † DEVIATION is the default for between-subjects factors, while POLYNOMIAL is the default for within-subjects factors.
  ** Default if the subcommand or keyword is omitted. 
  This command reads the active dataset and causes execution of any pending commands. See Command Order for more information. 
  
  Example 1
  * Analysis of Variance 
  MANOVA RESULT BY TREATMNT(1,4) GROUP(1,2).
  
  Example 2 
  * Analysis of Covariance 
  MANOVA RESULT BY TREATMNT(1,4) GROUP(1,2) WITH RAINFALL.
  
  Example 3 
  * Repeated Measures Analysis 
  MANOVA SCORE1 TO SCORE4 BY CLASS(1,2)
    /WSFACTORS=MONTH(4).
  
  Example 4 
  * Parallelism Test with Crossed Factors 
  MANOVA YIELD BY PLOT(1,4) TYPEFERT(1,3) WITH FERT
    /ANALYSIS YIELD
    /DESIGN FERT, PLOT, TYPEFERT, PLOT BY TYPEFERT,
     FERT BY PLOT + FERT BY TYPEFERT
     + FERT BY PLOT BY TYPEFERT.
  

• Why can't I just use multiple univariate ANOVA tests rather than MANOVA?
  Multiple univariate ANOVAs should not be used to "follow up" on a significant MANOVA. A significant MANOVA means that groups differ significantly on the canonical variates Put another way, examining group differences is best done by examining the canonical variates resulting from MANOVA. Examining group differences in terms of univariate ANOVAs involves a loss of power and increased chance of Type I error. See Huberty & Morris (1989). In general, MANOVA is conducted when it is expected that the resulting canonical variates reflect some construct differentiating the groups. If, however, it is known that the dependent variables are conceptually unrelated, then MANOVA may be inappropriate and univariate ANOVAs useful. Huberty & Morris (1989: 301) recommend MANOVA over multiple ANOVAs whenthe purpose is "to identify outcome variable system constructs, to select variable subsets, [or] to determine variable relative worth."

  With regard to repeated measures designs, multiple univariate ANOVA runs are not equivalent to the multivariate analysis done by a single MANOVA and will have less power (more chance of type II errors, which are false negatives, thinking there is no relationship when there is). In repeated measures designs, when contrasts of group means are correlated with each other, multiple univariate ANOVA tests may fail to reject the null hypotheses of no group differences. This is because ANOVA tests differences in means under assumptions of independence, whereas MANOVA is sensitive not only to mean differences but also does not require the assumption of orthogonality (lack of correlations) among the contrasts. Correlation of contrasts is tested by Mauchly's sphericity test, which, if significant, indicates a violation of the sphericity model (a model which implies uncorrelated contrasts) and thus requires the researcher to employ MANOVA rather than a series of univariate ANOVAs. In general, MANOVA is used to test group differences when more than two levels are involved in a repeated measures design because, unlike multiple univariate ANOVA, MANOVA does not require the assumption that contrasts be uncorrelated. When multiple univariate ANOVA test results differ from MANOVA results on the same data, this indicates that the contrasts between the levels of the repeated measures factors are correlated across subjects.

• How do I write up the results of my MANOVA analysis?
  The answer, of course, varies with the data. Consider a simple MANOVA in which a treatment group is exposed to a stimulus and there is a partial treatment group and a control group. Treatment could be exposure to a movie about recycling. Partial treatment could be exposure to a pamphet with the same information about recycling. The control group would receive no information. Thus there is a factor labeled Group which is the independent variable. Let the dependent variables be measures post-treatment perceptions of how likely the subject is on a scale from 1 to 10 (treated as an interval measure, or objective data might be obtained from actual behavior) to recycle Paper, Cans, and Plastics.

  The write-up would first state the results of the overall test of inter-group differences: "The Wilks Lambda multivariate test of overall differences among groups was statistically significant (p=0.032)." Depending on the publication, the parenthetical term might contain the F parameters and value (ex., "F(4,90) = 6.28"). Optionally, one might also state "The F statistic for Wilks' Lambda was exact.", which is reported under the relevant SPSS table. If there had been more than one factor, then the overall test is reported similarly for each factor. Also, one should always report effect size as well as significance. Before running the analysis, under the Options button, check "Estimates of effect size," which will cause partial eta-squared to be reported. State, "Although significant, the effect size of this relationship was weak as indicated by partial eta-squared = .11)."

  The "Tests of Between-Subjects Effects" table following the overall multivariate tests gives significance levels and partial eta-squared for each dependent. The write-up would state something like, "Univariate between-subjects tests showed that media exposure was significantly and moderately related to recycling paper (p=.0001; partial eta-squared = .21) and cans (p=.041; partial eta-squared = .10), but not to plastics (p=.421; partial eta-squared = .01)."

  Since the overall test when significant only shows at least one group is significantly different from another, the next research statement in the write-up must report post-hoc contrast tests among groups. For instance, in the table of "Multiple Comparisons" SPSS prints the mean difference on the dependent variable between any two groups and its corresponding significance. After using this test, the researcher might state, "Follow-up univariate post-hoc comparisons between groups using F statistics and Bonferroni-type simultaneous confidence intervals based on Student's t distribution also showed that the movie treatment group was significantly related to the recycling of paper (p=0.002) and cans (p=0.01) but not plastics (p=.055)." A similar statement would be made about the pamphlet group. The control group, which is the omitted category, would not have a corresponding statement. There are no partial eta-squareds to report. Optionally, one might list F parameters and values as well.

• How many dependents can I have in MANOVA?
  There is no theoretical limit, but keep in mind that as one increases the number of dependents there is a decline in interpretability, the likelihood of error based interactions increases, and there is a loss of power (that is, increased likelihood of Type II errors - thinking you don't have something when you do).

• Explain the syntax for MANOVA in SPSS.
  Note: This has largely been replaced by GLM syntax.
  MANOVA opinion1 opinion2 opinion3 BY EducLevel (1, 2) SESlevel (1, 3)
  / WSFACTORS = Ideology (3)
  / WSDESIGN = Ideology
  / PRINT = CELLINFO (MEANS)
  / DESIGN.

  Line 1: The MANOVA command word is followed by the three variables opinion1, opinion2, and opinion3. These represent the three levels of the within-subjects factor Ideology. The BY keyword tells SPSS that what follows are the groups or between-subjects factors; in this case, EducLevel and SESlevel. Following each of the two between-subjects factors are two numbers between parentheses. SESlevel (1,3) simply means that the variable SESlevel has three levels coded as 1, 2, and 3. One may have no grouping variable and thus no BY clause.
  Line 2: The slash mark indicates a subcommand. The WSFACTORS subcommand, tells SPSS that there is one repeated factor called Ideology and that it has three levels (matching the three opinion measurements listed after the MANOVA keyword). This is needed by SPSS to interpret the list of dependent variables in line 1. The WSFACTORS subcommand follows the MANOVA command when there is a within-subjects factor, which is to say when there is a repeated measures design.
  Line 3: The WSDESIGN subcommand tells SPSS to test the within-subjects hypotheses for repeated measures designs.
  Line 4: The PRINT subcommand specifies the output. CELLINFO (MEANS) prints cell means and standard deviations used to evaluate patterns in the data. Many additional statistics could be requested.
  Line 5: The DESIGN subcommand causes SPSS to test the between-subjects hypotheses.

  The general MANOVA syntax, from the SPSS manual, is:

  MANOVA
    depvarlist  [BY indvarlist (min,max) [indvarlist (min, max) ...]
    [WITH covarlist]]
    [/WSFACTORS = name (levels) name...]
  
    [/{PRINT | NOPRINT} = [CELLINFO [(MEANS SSCP COV COR ALL)]]
      [HOMOGENEITY [(BARTLETT COCHRAN BOXM ALL)]]
      [SIGNIF (MULTIV UNIV AVERF AVONLY EFSIZE ALL)]]
  
    [/OMEANS [VARIABLES(varlist)] [TABLES ([CONSTANT] [factor BY factor])]]
  
    [/CONTRAST(factor) = {POLYNOMIAL[(#)] | SPECIAL(k1s + contrasts)}]
    [/CONTR ...}
  
    [/WSDESIGN = effect ...]    [/DESIGN = effect ...]
  
  Notes on Effects
    Keywords: BY, W or WITHIN, MWITHIN
    Varname(#): # = one of k-1 contrasts or one of k levels
  
  

  Click here for syntax for other MANOVA commands for various designs.

• What is analysis of residuals for in MANOVA?
  A plot of standardized residual values against values expected by the MANOVA model tests the assumption of MANOVA that residuals are randomly distributed. If there are any observable systematic patterns, the model is questionable even if upheld by significance testing.

• Is there a limit on the number of covariates which can be included in an multiple analysis of variance?
  Not really. Whereas SPSS limits you to 10 in ANOVA, the limit is 200 in MANOVA -- more than appropriate for nearly all research situations. As one adds covariates, the likelihood of collinearity of the variables increases, adding little to the percent of variance explained (R-squared) and making interpretation of the standard errors of the individual covariates difficult.

• What is step-down MANOVA?
  Step-down MANOVA, also called the Roy-Bargman Stepdown F test, is a conservative approach to significance testing of the main effects, designed to prevent the inflation of Type I errors (thinking you have something when you do not). Stepdown tests perform a univariate analysis of the significance of the first dependent, then test the second dependent controlling for the first as a covariate, and so on, sequentially rotating the dependents to the status of covariates. This process continues until one encounters a significant F for a variable, leading the researcher to conclude it is significantly related to the independent (classification) variables over and above the "covaried out" previously-tested dependent variables. Step-down MANOVA is recommended only when there is an à priori theoretical basis for ordering the dependent (criterion) variables. The order of the variables is critical to the results obtained by this method. In SPSS syntax mode, specify PRINT SIG(STEP) to obtain the stepdown F test. As of SPSS Version 10, it was not available in menu mode but using GLM one could obtain identical results by conducting multiple runs, each time with a different dependent as covariate.

  In SPSS, make sure that the dependents are entered in the desired order, then in the MANOVA syntax, enter PRINT SIGNIF(STEPDOWN) or simply PRINT SIG(STEP). For example:

  manova
     var1 var2 var3 BY gender(1,2)
     /print signif(stepdown)
    ...
  

  See James Stevens, Applied Multivariate Analysis for the Social Sciences, 2nd Edition.

• What is the "protected F" or least significant difference (LSD) test in MANOVA? How does it relate to the use of discriminant analysis in MANCOVA?
  In the second step on MANOVA, when one tests for specific group differences, having established the overall difference among groups using the F test, today the most common method relies on multiple discriminant analysis (MDA) and associated significance tests (ex., Wilks, Hotelling, Pillai-Bartlett, etc.) as discussed above. However, the earlier method, of following a significant MANOVA with a series of ANOVAs on each of the dependent variables, is still used. This traditional method is called the protected F test, the protected t test, or the least significant difference test. Using multiple univariate ANOVA tests at a nominal alpha significance level (ex., .05) is misleading -- the actual significance level will be much higher (that is, > .05), affording less protection against Type I errors (thinking you have something when you don't) than the researcher may be assuming. For this reason, the protected F or LSD method is no longer recommended.

  Using discriminant analysis, the MANCOVA dependents are used as predictor variables to classify a factor (treatment) variable, and the discriminant beta weights are used to assess the relative strength of relation of the MANOVA dependents to the factor. The beta weights indicate the strength of relation of a given dependent controlling for all other dependents

Bibliography

• Bray, James H. and Scott E. Maxwell (1985). Multivariate analysis of variance. Quantitative applications in the social sciences series #54. Thousand Oaks, CA: Sage Publications.
• Gill, Jeff (2001). Generalized Linear Models: A Unified Approach. Thousand Oaks, CA: Sage Publications. Seires: Quantitative Applications in the Social Sciences, No. 134. A mathematical overview of GLM.
• Hand, D.J. and C. C. Taylor (1987). Multivariate analysis of variance and repeated measures. London: Chapman and Hall.
• Huberty, Carl J. & Morris, John D. (1989). Multivariate analysis versus multiple univariate analyses Psychological Bulletin, 105(2), 302-308.
• Huyn, H. and G. K. Mandeville (1979). Validity conditions in a repeated measures design. Psychological Bulletin, Vol. 86: 964-973.
• Nelder, J. A. and R. W. M. Wedderburn (1972). Generalized liner models. Journal of the royal Statistical Society, A, 135: 370-384. This is the seminal article on GLM.
• Nichols, David (1993). Interpreting MANOVA parameter estimates. SPSS Keywords. February. Retrieved 3/5/2009 from http://support.spss.com/ProductsExt/SPSS/Documentation/Statistics/articles/paramest.htm.
• Olson, C. L. (1976). On choosing a test statistic in multivariate analyses of variance. Psychological Bulletin, Vol. 83: 579-586. Olson's tests showed Pillai-Bartlett trace to be more robust than Wilks's U.