Multivariate Analysis of Variance (MANOVA)
MANOVA
NOTES
Using DVs in combination through MANOVA circumvents this high experimentwise error rate. Then ANOVA is used to identifymain source of these effects. Researcher is mainly interested inresults ofANOVA
MANOVA combines DVs in diff ways so as to maximizeseparation ofconditions specified by each comparison indesign. – designs including a within-subjects factor having more than 2 levels must meetsphericity assumption. However,most common within-subject factor in applied research is time of testing (pretests and posttests) which clearly violate this assumption. key research questions addressed by MANOVA are very muchsame as those addressed by an ANOVA design. If participants are assigned at random toexperimental conditions indesign, MANOVA allows a researcher to examinecausal influence ofinDVs on a set of DVs both alone (main effects) and in interaction.
With equal numbers of participants in each cell ofdesign, these main effects and interactions are independent of one another makinginterpretation ofresults very clear.Amount of vari. In DVs explained by IVs is important to know and can be determined using MANOVA. MANOVA involves testinginfluence ofinDVs upon a set of DVs. Therefore, it is possible to examine which of these DVs is most impacted by these inDVs. That is,relative impact of IVs on each DV can be estimated.
MANOVA (like ANOVA) can also be extended to include covariates (MANCOVA). That is,influence ofinDVs onDVs controlling for important covariates can be assessed. Hypotheses often specify particular contrasts within an overall interaction such as a comparison of treatment group with a placebo control group and a no-treatment control group at posttest. MANOVA allows tests of these specific hypotheses as well asoverall main effects and interactions.
When to use MANOVA
On page 251, TF argue that MANOVA should be used whenDVs are independent of one another. In this situationvarious diff impacts ofinDVs is most likely to be completely assessed. As well, they argue that when DVs are highly correlated, it might be more advisable to combine these vars into a composite and use a simpler anal.. However, this position does not seem to addresssituation most likely to be faced by researchers using MANOVA; namely, that a set of DVs measuring diff but related constructs are employed to examineeffects ofinDVs (usually DVs are moderately correlated). common research strategy used by experimental and applied psychologists alike: strategy in which sets of DVs bracketing a general but loose construct are entered into separate MANOVA’s. For ex., indicators of psychological distress might beDVs in one anal., while indicators of physical health problems might beDVs in another. Each MANOVA can then give information onDV(s) in each set that is most affected bytreatment (experimental manipulation).
Multivariate Normality andDetection of Outliers
Multivariate ANOVA procedures makeassumption thatvarious means in each cell ofdesign and any linear combination of these means are normally distributed. Provided there are no outliers in each cell ofdesignanal. Is robust to this assumption, especially sinceCentral Limit Theorem states thatdistribution of samp. Means derived from a non-normal distribution of raw scores tends to be normal. As a rule of thumb, applied researchers try to achieve fairly equal numbers of participants within each cell ofdesign and a minimum with-cell samp. Size of at least 20 (I prefer 20 plusnumber of DVs) so as to ensure that this assumption is not seriously violated. However,most important guideline is to check each cell for outliers before running any MANOVA.
Homogeneity ofVari.-Covari. Matrix
In MANOVAequivalent assumption is thatvari. – covari.
matrix forDVs within each cell ofdesign is equal. It is important to remove or correct for outliers before checking this assumption as they greatly influencevalues invari. – covari. Matrices. Ifcell sizes are relatively equal andoutliers have been dealt with,anal. Is robust to this assumption.
If cell sizes are unequal, use Box’s M test ofhomogeneity ofvari. – covari. Matrices. This test tends to be too sensitive and so Tabachnick and Fidell recommend thatresearcher only be concerned if this test is sig atp < .001)=”” level=”” and=”” cell=”” sizes=”” are=””>
As well, if larger cell sizes are associated with larger vari.S and covari.S,sig levels are conservative. It is only when smaller cell sizes are associated with larger vari.S thattests are too liberal indicating some effects are sig when they really are not (a high Type 1 error rate). Fmax isratio oflargest tosmallest cell vari.. An Fmax as large as 10 is acceptable provided thatwithin-cell samp. Sizes are within a 4 to 1 ratio. More discrepant cell sizes cause more severe problems.
Unequal Cell Sizes homogeneity ofvari. – covari. Matrices can not be tested ifnumber of DVs is greater than or equal tonumber of research participants in any cell ofdesign assumption is easily rejected andpower ofanal. Is low whennumber of respondents is only slightly greater thannumber of DVs. This can result inMANOVA yielding no sig effects, even though individual ANOVAs yield sig effects supportinghypothesis,
rule of thumb of 20 plusnumber of DVs as a minimum cell size andstrategy of analysing small sets of DVs that often measure a general, loosely defined construct (e.G., indicators of psychological distress).
observed power” value which isprobability thatF value for a particular multivariate main effect or interaction is sig ifdifferences amongmeans inpop is identical todifferences amongmeans insamp.Non-experimental designs in which samp. Sizes reflectrelative sizes ofpops from which they are drawn (their relative importance), Method 2 is used in which there is a hierarchy for testing effects starting withmain effects (and covariates) which are not adjusted, then2 way interaction
terms which are adjusted formain effects, etc… This method is labelled METHOD = SEQUENTIAL Linearity
Multicollinearity
AlthoughDVs are intercorrelated, it is not desirable to have redundancy among them MANOVA analyses outputpooled within-cell correlations amongDVs. As well, MANOVA prints outdeterminant ofwithin-cell vari. -covari. Matrix which Tabachnick and Fidell suggest should be greater than .0001. If these indices suggest
Homogeneity of Regression and Reliability of Covariates
When covariates are used inanal.
(MANCOVA
Or ifresearcher plans to useRoy-Bargmann stepdown procedure to examinerelative importance ofindividual DVs,relationship betweenDVs andcovariates MUST besame within every group indesign. That is,slope ofregression line must besame in every experimental condition. If heterogeneity of regression is found, thenslopes ofregression lines differ; that is, there is an interaction betweencovariates andinDVs. If this occurs, MANCOVA is an inappropriate anal. Strategy to use.
Before running a MANCOVA orRoy-Bargmann procedure, therefore,pooled interactions amongcovariates andinDVs must be shown to be non-sig (usually, p < .01)=”” is=”” used=”” to=”” detectsig=”” of=”” these=”” pooled=”” interaction=”” terms). =”” in=”” addition,covariates=”” must=”” be=”” reasonably=”” reliable=”” (alpha=””> .80) Using unreliable covariates can results ineffects ofinDVs onDVs being either under-adjusted or over–adjusted
Other Criteria for Statistical Sig
Most usually researchers useapproximate F derived from Wilks’ Lambda ascriterion for whether an effect in MANOVA is sig or not. However, three other criteria are also used andSPSS MANOVA and GLM programs output all four
.
These criteria are equivalent wheneffect being tested has one degree of freedom, but are slightly diff otherwise because they create a linear combination ofDVs that maximizesseparation ofgroups in slightly diff ways Pillai’s Trace is derived by extracting eigenvalues associated with each main effect and interaction indesign. Like factor anal., larger eigenvalues correspond to larger percentages of vari. Account for by these effects. Pillai, therefore, derived an approximate F test to testextent to which these eigenvalues are unlikely to occur by chance ifnull hypothesis is true Pillai’s Trace is an important statistic because it is more robust toassumption of homogeneity ofvari. – covari. Matrix, especially when there are unequal n’s incells ofdesign. When there are prob w/ design, use Pillai’s as it ismore conservative andmore robust test. Otherwise use Wilk’s Lambda.
OUTPUT FOR GLM SIMPLE MANOVA
Descriptive Statistics
Note: This shows the cell means, the marginal means, and the grand mean (with variances and n).
Box’s Test of Equality of Covariance Matrices
Tests the null hypothesis that the observed covariance matrices of the dependent variables are equal across groups.
Note: Box’s M tests homogeneity of the variance – covariance matrix.
Multivariate Tests
Note: The F values for the main effect for intensity and the interaction derived from Wilks’ Lambda are slightly larger than those derived from Pillai’s Trace. Eta squared (actually partial 2) is an estimate of the percentage of variance accounted for by the main effects and interactions (tend to be an overestimate). The observed power column is the power of the design to detect population differences among the means in the main effect or interaction that are identical to the differences among the means found in the sample
Levene’s Test of Equality of Error Variances
Note: This the test of the homogeneity of variance assumption for each dependent variable separately. The analysis is robust to this assumption.
Tests of Between-Subjects Effects
Note: This table summarizes the univariate analyses of variance on each dependent variable. Notice that the main effect for Method is significant for Speed but not for Accuracy. However, the multivariate test indicates that this main effect is not significant (or marginally significant, p <>
Between-Subjects SSCP Matrix
Note: With these matrices you can calculate the significance of the effects in the design.
Residual SSCP Matrix
These are pooled within-cell matrices. The pooled within-cell correlation matrix shows that the two dependent variables are correlated quite highly across the cells of the design (r = .43). Notice that the residual SSCP matrix is the same as the error SSCP matrix above. Its determinant is (1333 x 185 – 2112) = 0.202 x 106. To calculate Wilks’ Lambda, the determinant of this SSCP matrix is divided by the determinant of a matrix created by adding the error matrix to the effect matrix:
For example, the denominator of for the Intensity Main effect is:
1055 1112 1333 211 2388 1323
+ =
1112 1177 211 185 1323 1362
The determinant of this matrix is (2388 x 1362 – 13232) = 1.503 x 106. = 0.202 x 106 / 1.503 x 106 = 0.134 (see previous table).
The MANOVA program OUTPUT will produce similar output (same information, different format) using the following syntax. In this output, the MANOVA program gives the determinant of the pooled within-cell variance – covariance matrix which in this case is 69.14.
Sometimes a researcher wants to test specific hypothesized contrasts.
This is most easily done through the MANOVA program by specifying a set of orthogonal contrasts equal to the number of degrees of freedom in the overall main effect or interaction.
The output gives the multivariate tests and the univariate tests for each dependent variable (t tests) for these specific contrasts within the overall main effect for Intensity. For example, for the second contrast the output looks like:
Note.. F statistics are exact
EFFECT .. INTENSIT(2) (Cont.)
Later in the output, the computer prints out the statistics for the univariate test for the specific contrasts tested.
Estimates for SPEED — Individual univariate .9500 confidence intervals
This output shows that the difference between the marginal means for the dependent variable, Speed, comparing the Intensity Conditions 3 hours for 4 weeks versus 4 hours for 3 weeks is:
Difference = 34.0 – 28.3 = 5.7 (see marginal means in an earlier table) This difference is significantly different from zero showing that typing speed is faster in the 3 hours for 4 weeks condition: t (54) = 3.63, p < .001=”” with=”” the=”” 95%=”” confidence=””> >Note that t2 = F = 13.16 In the same way, the difference in average typing speed between the 2 hours for six weeks condition and the remaining conditions is:
Difference = 2 x 38.55 – (34.0 + 28.3) = 14.8 The output (not shown) indicates that this contrast is also significant, t (54) = 5.43, p < .0001),showing=”” typing=”” speed=”” is=”” greater=”” in=”” the=”” low=”” intensity=”” condition=”” than=”” in=”” the=”” other=””> >
EXAMPLE OF A SIMPLE
MANCOVA
Before a MANCOVA can be conducted, the homogeneity of regression assumption must be checked. This is achieved by specifying the variable, SEX, as a covariate and then running a MANOVA in which SEX is one of the independent variables. If the assumption of homogeneity of regression within each cell of the design is not correct, then SEX will interact with the independent variables in the design. Therefore, the MANOVA is used to test the pooled effect of all of these interactions (after entering the main effect for SEX first).
EFFECT .. SEX BY METHOD + SEX BY INTENSIT + SEX BY METHOD BY INTENSIT
Multivariate Tests of Significance (S = 2, M = 1 , N = 22 1/2)
Note.. F statistic for WILKS’ Lambda is exact. -This test shows that the pooled interactions between the covariate and the remaining factors in the design are not significant (remember p < .01)=”” is=”” the=”” criterion). =”” therefore,=”” the=”” assumption=”” of=”” homogeneity=”” of =”” regression=”” has=”” been=””>
If two or more covariates are used, their effects are pooled and then these pooled effects are examined in interaction with the independent variables.
After the homogeneity of regression assumption has been checked, the researcher is ready to conduct the MANCOVA using the following syntax:
first part of the output displays the cell and marginal means, standard deviations, and “n” for the two dependent variable followed by a similar table for the covariate
Then the output gives tables showing the SSCP matrix, the variance –covariance matrix, and the correlation matrix among the dependent variables and the covariate for each cell of the design are displayed at this point that the output gives the pooled within-cell variance – covariance matrix and its determinant as shown below:
Pooled within-cells Variance-Covariance matrix
Determinant of pooled Covariance matrix of dependent vars. = 69.14202 LOG(Determinant) = 4.23616
– This output shows that there is no multicollinearity problem in this data set.
The output then shows the homogeneity of variance – covariance test.
Multivar test for Homogen of Dispersion matrices
-This stat shows assumption of homogeneity of vari – covari across the cells in design is tenable.
Now the adjusted SSCP matrices are given:
Adjusted WITHIN CELLS Correlations with Std. Devs. On Diagonal
–
This matrix shows that the two dependent variables correlate less when SEX is partialled out of their relationship (r = .239)
Now the computer prints out the adjusted within-cell SSCP matrix Adjusted WITHIN CELLS Sum-of-Squares and Cross-Products
–This is the pooled SSCP matrix adjusted for the covariate, SEX.
The pooled SSCP matrix and correlation matrix for the MANOVA without any covariates are shown below. Note that the effect of the covariate is to make the values in the SSCP matrix smaller and to reduce the correlation among the dependent variables from .426 to .239 (partialling out the covariate).
Residual SSCP Matrix
The significant effect of the covariate is now given:
EFFECT .. WITHIN CELLS Regression
Multivariate Tests of Significance (S = 1, M = 0, N = 25 )
Note.. F statistics are exact
EFFECT .. WITHIN CELLS Regression (Cont.)
Univariate F-tests with (1,53) D. F
-This part of the analysis shows that the effect of the covariate is significant (it is an effective covariate). If the covariate is not significant, there is no need to perform a MANCOVA!
-The remaining analyses show both the multivariate and univariate main effects and interactions for the independent variables:
EFFECT .. METHOD BY INTENSIT
Multivariate Tests of Significance (S = 2, M = -1/2, N = 25 )
Note.. F statistic for WILKS’ Lambda is exact.
EFFECT .. METHOD BY INTENSIT (Cont.)
Univariate F-tests with (2,53) D. F
Note that adjusting for the covariate results in a significant main effect for method of instruction; L= 0.876, F(2, 52) = 3.69, p <> This effect was only marginally significant in the MANOVA; L = 0.908, F(2, 53) = 2.69, p < .08. =”” the=”” effect=”” size=”” for=”” a=”” main=”” effect=”” or=”” interaction=”” (after=”” covarying=”” out=”” sex),=”” partial=”” eta2=”” ,=”” can=”” be=”” calculated=”” through=”” the=”” formula:=”” partial=”” eta2 =” ” 1=”(L)1/s,” but=”” it=”” is=”” easier=”” just=”” to=”” repeat=”” the=”” analysis=”” using=”” glm=”” and=”” reading=”” partial=”” eta2=”” from=”” the=””>
Assessing the Influence of the Independent Variables on the Individual Dependent Variables
Once the MANOVA has identified significant main effects and/or interactions, the researcher will usually want to know which of the set of dependent variables is most affected by the independent variables. Most usually, researchers will look for significant univariate tests of these effects for each dependent variable using a Bonferroni adjustment so that the Type 1 error rate is not inflated. For a set of p dependent variables, this adjustment is:
= 1 – (1 – 1)(1 – 2)(1 – 3)…. (1 – p)
-this adjustment assumes that the effects obtained are independent of one another which is clearly not the case when the dependent variables are intercorrelated. Nevertheless, this is still the most common way of interpreting and reporting the results of a MANOVA and the one we will use in this class! Tabachnick and Fidell recommend that researchers give the pooled within-cell correlation matrix among the dependent variables if a researcher adopts this analysis strategy.
–
Another way to overcome this problem is Roy – Bargmann stepdown analysis in which the researcher specifies a sequence of dependent variables in order of importance. Then, he or she conducts an ANOVA on the dependent variable of most importance, an ANCOVA adjusting for the effects of the first dependent variable on the second most important dependent variable, and so on. As the successive ANCOVAs are independent of one another, the Bonferroni correction is an accurate adjustment which controls the Type 1 error rate. However, the limitation to using this analysis is that the researcher must be able to clearly specify the order in which to enter the dependent variables into the analysis. This is usually a hard, if not impossible task given the nature of current psychological theories
-A third way to approach this problem is to use the loading matrix (raw discriminant function coefficients) from a Discriminant Function Analysis output which is obtained through SPSS MANOVA to identify those dependent variables that correlate highly with the linear combination of dependent variables (the discriminant function) that achieves the maximum separation of the groups specified by a main effect or interaction
A Priori Power Analysis (MANOVA)
Step 1: Create a dummy data set with 2 subjects in each condition such that the mean score in each condition on each dependent variable is the mean you anticipate getting if your hypothesis is correct (based upon past research findings). For example, consider the case where you want to compare the two instructional typing methods (1 = traditional versus 2 = motivational) and you expect Speed to average 32 for method 1 and 35 for method 2; accuracy to average 18 for method 1 and 20 for method 2 based upon past research, etc…
Run a dummy MANOVA to obtain the data in a matrix form:
Save this data matrix (as a SAV file).
Now change the values in the matrix to be the same as those obtained in past research studies. That is, change the standard deviations of the DVs and their inter-correlation(s).
To obtain power estimates, do several runs with this adjusted matrix using various cell sizes to find the cell size that gives you adequate power.
Using the example in the lecture notes, the correlation between speed and accuracy is 0.40, and the overall SD for speed is 7.0 and for accuracy is 5.0.
Therefore, with N = 60 the matrix is changed to:
The subcommand “/power= f(0.5) exact” tells the computer to calculate the power if the probability of a type I error is set at p < .05)the=”” word=”” “exact”=”” means=”” that=”” the=”” computer=”” does=”” an=”” exact=”” calculation=”” of=”” power=”” rather =”” than=”” a=”” much=”” quicker=”” and=”” rougher=”” approximate=”” calculation=””>
The Key Output from this run is:
EFFECT .. METHOD
Multivariate Tests of Significance (S = 1, M = 0, N = 27 1/2)
This shows the power to detect the anticipated mean differences between methods using Speed and accuracy as the DVs is 0.86 if you use MANOVA.
Increasing the n to 50 per cell, increases the power of this MANOVA analysis to 0.94 as shown below.
EFFECT .. METHOD
Multivariate Tests of Significance (S = 1, M = 0, N = 47 1/2)
The Mathematical Basis of MANOVA L = Lambda hsquare = eta square (strange n sign) S = variance from sum of squares matrix The mathematical basis of MANOVA is explained by extension using the mathematical basis of ANOVA. Given that there is more than one dependent variable in MANOVA, this analysis includes the SSCP matrix, S, among these dependent variables. Significance tests for the main effects and interactions obtained through the MANOVA procedure compare ratios of determinants of the SSCP matrices calculated from between group differences and pooled within-group variability. (Directly analogous to calculating the F ratio by dividing the mean square between by the mean square within in ANOVA.) The key point to grasp here is that the determinant of a SSCP matrix can be conceptualised as an estimate of the generalized variance minus the generalized covariance in this matrix. To show this consider the determinant of a correlation matrix for two dependent variables:
1 r12
= (1 – r122)
r12 1
As this example clearly shows, the determinant is the proportion of the variance not common to these two interrelated variables (see also Appendix A, p. 932 for a different but equivalent explanation using a variance – covariance matrix). A difference from ANOVA is that the significance of each effect calculated by the MANOVA procedure is for a linear combination of the dependent variables that maximizes the differences among the cells defined by that effect. This means that each effect is associated with a different linear combination of the dependent variables that satisfy this criteria and knowing how the dependent variables are weighted for each significant effect is of interest to the researcher. The implication is that the proportion of variance accounted for by all the effects combined is greater than 1 (the upper limit for a correlation and a square multiple correlation) making it difficult to know exactly how much variance is accounted for by each of the significant effects identified through MANOVA, although their relative strengths are known. In an ANOVA of a between-subjects factorial design, the total sum of squares can be partitioned into the sum of squares associated with each main effect and interaction (more generally into orthogonal contrasts) and the pooled within-cell sum of squares. With the exception of the last sum of squares, these are derived from the deviations of mean scores from the grand mean. Estimates of variance are then derived from each of these sum of squares (the mean squares) and the significance of the effect is tested by examining the ratio of each of the various between-groups mean squares with the pooled within-groups mean square (the F ratio). In an analogous manner, a MANOVA of a between-subjects factorial design uses the SSCP matrix, S, for each effect in the design which are derived by post multiplying the matrix of difference scores for the effect (mean – grand mean) by its transpose. The determinants (the generalized variance) of these matrices can then be used to test whether the effect is significant or not. To illustrate this process, consider a simple 2 x 2 between-subjects design with 10 participants in each condition who answered 3 dependent variables (for a complete worked example, see TF section 7.4.1, pp. 255-263). Then the matrix of difference scores between the mean and the grand mean on each DV for the first factor, A, is:
Levels of Factor A
A1 A2
DV1 M11 – MG1 = m11 M12 – MG1 = m12
DV2 A = M21 – MG2 = m21 M22 – MG2 = m22
DV3M31 – MG3 =m31 M32 – MG3 = m32
where M11 is the marginal mean for the first dependent variable at the first level of A (A1), M12 is the marginal mean for the first dependent variable at the second level of A (A2), etc…, and MG1
is the grand mean for the first dependent variable, etc…
A is a 3 (dependent variables) by 2 (levels of factor A) matrix and A AT
is the 3 x 3 SSCP matrix summing over the two levels of factor A.
SA = SSCPA = A . AT = a 3 x 3 SSCP matrix for factor A which gives the
sum of squares and cross products for the deviations of the marginal mean values for factor A around the grand mean for all three dependent variables (summing over levels of factor A).
m112 + m122 m11m21 + m12m22 m11m31 + m12m32
=m21m11 + m22m12 m212 + m222 m21m31 + m22m32
m31m11 + m32m12 m31m21 + m32m22 m312 + m322
The means are based upon the scores of 20 participants, so this matrix is multiplied by 20 to estimate the sum of squares associated with the main effect for factor A. The determinant of this matrix is an estimate of the generalized variance associated with this main effect (the sum of the squares minus the sum of the cross products). The diagonal elements in this matrix are the sum of the squares for each of the three dependent variables (summed across levels of factor A). In a similar fashion, the generalized variance associated with the main effect for B can be derived as can the generalized variance for the interaction between A and B (The SSCP matrix for the four cells of the design is estimated, SSCPA, B, AB, and the
matrices for the 2 main effects are subtracted from it to give the SSCP for the interaction term alone, SSCPAB.)
Finally, the avg within cell SSCP matrix is estimated.
All these matrices are symmetrical square matrices with an order determined by the number of dependent variables in the analysis. Therefore, they can be added to one another. In particular, the SSCP within cell error matrix can be added to each of the matrices associated with the main effects and interactions (or, more generally, to any SSCP matrix derived from a contrast). When this is done, a statistics called Wilks’ Lambda, L, can be calculated as follows:
Lambda = Serror / Seffect + Serror
This statistic can be converted into an approximate F test which is outputted by the computer along with its degrees of freedom and statistical significance. This approximate F is part of the SPSS output, but it can be calculated by hand using the following formulae:
Approximate F (df1, df2) = (1 – y)(df2) / y (df1)
Given that there are p dependent variables, y = (L )1/s
where s = [(p2 dfeffect2 – 4) / (p2 + dfeffect2 – 5)]1/2
df1 = p dfeffect
df2 = s [ dferror – (p – dfeffect + 1) / 2] – [ (p dfeffect – 2) / 2 ]
In analysis of variance, the proportion of the variance accounted for by each main effect and interaction can be calculated. Similarly, in MANOVA the proportion of variance accounted for by the linear combination of the dependent variables that maximizes the separation of the groups specified by a main effect or an interaction is simply:
h2 = 1 – L, remembering that Wilks’ Lambda is an index of the proportion of the total variance associated with the within cell error on the dependent variables.
However, the h2values for any given MANOVA tend to yield high values that sum to a value greater than 1. Therefore, TF recommend a more conservative index:
Partial h2 = 1 – (L)1/s
where s = [(p2 dfeffect2 – 4) / (p2 + dfeffect2 – 5)]1/2
The Mathematical Basis of MANCOVA
In MANOVA, the covariates are initially considered as one of the dependent variables. However, the resulting SSCP matrix is partitioned into smaller matrices that can be used to adjust the dependent variables for the effects of the covariates. matrix for the main effect of A in the 2 x 2 between-subjects factorial design discussed earlier:
SA = SSCPA = A . AT
m112 + m122 [m11m21 + m12m22 m11m31 + m12m32]
= [m21m11 + m22m12] m212 + m222 m21m31 + m22m32
[m31m11 + m32m12] m31m21 + m32m22 m312 + m322 If the first dependent variable is the covariate, then this matrix has three components: The SSCP matrix for the dependent variables, S(y), (the bottom right 2 x 2 matrix), the sum of squares of the covariate, S(x), (the top left term in the matrix – when there is more than 1 covariate, this is the SSCP matrix among the covariates), and the cross product terms between the covariate and the dependent variables, S(yx) (shown in square brackets).
Specifically:
S(y) = m212 + m222 m21mA31 + m22m32
m31m21 + m32m22 m312 + m322 (The unadjusted SSCP matrix for the DV’s)
S(x) = m112 + m122 (The sum of squares for the covariate. For more than one covariate, this would be a SSCP matrix).
S(yx) = m21mA11 + m22mA12
m31mA11 + m32mA12 (the cross product matrix between the covariate and the DV’s To obtain the SSCP matrix for the two dependent variables adjusting for the covariate, S* the following matrix equation is used:
S* = S(y) – S(yx) . (S(x))-1 (2 x 2) = (2 x 2) – (2 x 1) (1 x 1) (1 x 2) This adjustment is applied to the SSCP matrices for all the main effects and interactions in the design as well as to the SSCP matrix for the pooled within-cell error. Then the test for significance of the effect using (say) Wilks’ Lambda is applied to the adjusted SSCP matrices as well as to the covariates (which should be significant if they are effective). Note that if this covariate correlates with the dependent variables, this matrix subtraction results in a SSCP matrix with much smaller values.
SplitPlot Designs using MANOVA
In many applied research projects that evaluate social programs, a SP or mixed design1 (a design containing both between-subject factors and within-subject factors) is used in which a treatment grp and a control grp are compared over time from pretest to posttest to follow-up. The problem with analysing this design using ANOVA is that often the sphericity assumption is not met. Simply put, this assumption is that the correls between any two levels of the within-subjects factor are the same. Clearly, this is not the case when the time of testing spreads out over several months; 4 example, the correl between the pretest and a one year follow-up is almost necessarily less than the correl between the pretest and a posttest taken during the last treatment session. In PSY 805 you have learned how to correct the results of a repeated measures ANOVA when this assumption is not met using, 4 example, the Huynh – Feldt correction. However, another option is to use a special 4m of MANOVA called Profile Anal. Which does not require making the sphericity assumption at all.
1 A mixed design also refers to a design which includes both manipulated and non-manipulated (e.G., personality trait or housing type) var. This is why I prefer to use the term SP design.
Profile anal. Was developed in order to compare the response of diff grps (the between-subjects factor) across the levels of the within-subjects factor (or factors) on the same DV. Applying this to the applied research design just described, the profile anal. Tests whether the treatment and control grps differ across time (the levels hyp – tested by the main effect 4 the between-subjects factor), whether the intervention affects the treatment grp (the parallelism hyp – tested by the interaction between the between-subjects factor and the within-subjects factor), and whether there are historic, maturational or other systematic trends over time 4 the treatment and the control grps (the flatness hyp – tested by the main effect 4 the within-subjects factor).
In order to avoid the sphericity problem, the scores on adjacent levels of the within-subjects inDV are subtracted from one another creating segment scores which are used as multiple (and correlated) DVs in a one-way MANOVA. The average of these segment scores within any cell of the design represents the relationship between the DV and the two levels of the original within-subjects inDV which create the segment score (a slope). There4e, if the multivariate test is sig., the slopes representing the relationship between the DV and the within-subjects inDV are not parallel (the parallelism hyp). The implication is that the interaction between the between-subjects factor and the original within-subjects factor is sig., and this interaction is tested under the assumptions governing any multivariate anal. Of vari which do not include the sphericity assumption.
The Criteria 4 Statistical Sig. In Profile Anal
In this design, there is only one DV, so the between-subjects factor is tested using a univariate F test. The interaction between the between-subjects factor and the within-subjects factor is then tested in a one-way MANOVA using the same multivariate tests reviewed earlier, but the anal. Is done on the
segment scores rather than the original raw scores in the data set (the parallelism test). That is, Wilkes Lambda is calculated and tested using an approximate F test. The percentage of vari explained by the linear combination of the segment scores, partial 2, can also be calculated if the effect is sig..
The main effect 4 the within-subjects factor is then computed using the combined segment scores across grps as multiple DVs. As Tabachnick and Fidell point out (p. 324), this multivariate test examines whether the average slope of the segments across grps is sig.Ly diff from zero (flatness) and this can be examined using Hotelling’s T2. However the SPSS program converts this statistic into those reviewed earlier so I will not cover it in this class.
Remember that a diff linear combination of the DVs defined by the segments is used to test the main effect (flatness) and the interaction (parallelism); linear combinations that maximize the separation of the grps involved and that correlate with one another.
EXAMPLE OF A SIMPLE SP DESIGN USING MULTIVARIATE ANAL. OF VARI
Consider the following hypothetical data set in which a treatment grp and a control grp (GRP) are tested at pretest, posttest, and follow-up (TIME). The treatment grp consists of 30 senior executives from a large potash company who receive a new and intensive 4m of leadership training over a period of one month, whereas the control grp of 20 senior executives are placed on a wait list (they take the program in one year’s time). Then the leadership skills of these two grps are assessed at pretest, posttest, and, one year later, at follow-up.
This is a 2 (grp) x 3 (time) SP design (also called a mixed design) with one between-subject factor and one within-subject factor. One way to analyse this design is to use repeated measures anal. Of vari. However, the problem is that the sphericity assumption is almost certainly violated and, because the test is NOT robust to this assumption, the results are too liberal (more likely to contain Type 1 errors than the sig. Level would suggest). There4e, applied researchers often use Profile Anal. To analyse this design.
Notice that the three DVs in this anal. Are the three levels of the repeated measures factor (TIME). TIME is then specified as a factor in a profile anal. On the /WSFACTOR = TIMES(3) subcommand. The
RENAME subcommand is often used (you don’t have to use it) to clarify that the anal. Is not being done on the original DVs, but on segments. Three segments are specified. The first, C, is a constant and allows the main effect 4 the between subjects factor to be tested using the scores on the DV averaged across the levels of the repeated measures factor. The other two segments compare conditions using orthogonal contrasts. This is always the last subcommand when using MANOVA. Output results of the MANOVA using segment scores and the univariate SP ANOVA results 4 main effects and interactions involving the within-subjects factor are printed together.
This allows the researcher to read the output and decide which results to report: If the sphericity assumption is not violated, then he or she uses the results of the univariate tests; if this assumption is violated, then he or she uses the MANOVA results.
Cell Means and Standard Deviations
Variable .. PRE
Note: This output is produced by CELLINFO. Notice that sphericity is almost certainly violated (as expected) because the pretest and posttest correlate much more strongly with each other than with the follow-up.
Cell Number .. 2
Correl matrix with Standard Deviations on Diagonal
Note: Again the correls show the sphericity assumption is likely violated.
Pooled within-cells Vari-Covari matrix
Note: The determinant shows that there is no problem with multicollinearity among the DVs.
Multivariate test 4 Homogeneity of Dispersion matrices
Note: This test shows that the homogeneity of the within-cell vari – covari matrices is not violated (unequal n’s AND p < .001=”” is=”” the=””>
Orthonormalized Trans4mation Matrix (Transposed)
Note: The trans4mation matrix shows how the computer has 4med the orthogonal contrasts using segments created from the original three levels of the repeated measures factor (this is what is meant by orthonormalized). Segment 1 compares the pretest with the follow-up, whereas segment 2 compares the posttest with the average of the pretest and follow-up. Diff sets of orthogonal contrasts can be created and the MANOVA would still give the same overall results 4 the repeated measures main effect and interaction.
Tests of Between-Subjects Effects
Note: This is the main effect 4 GRP which shows that the treatment grp has more leadership skills than the control grp, but it must be interpreted in the context of the sig. Interaction. (The marginal means 4 GRP are not given as part of this output).
WITHIN+RESIDUAL Correls with Std. Devs. On Diagonal
Note: The average within-cell correl between the segments is 0.593. These are the DVs used by MANOVA to test the TIME main effect and the GRP x TIME interaction.
Tests involving ‘TIME’ Within-Subject Effect
Note: This is the test 4 the sphericity assumption. In this instance the assumption has been violated if p < .05)=”” indicating=”” that=”” the=”” researcher=”” should=”” interpret=”” the=”” results=”” of=”” the=”” manova=”” tests=”” rather=”” than=”” the=”” sp=”” anova=”” tests=”” (see=”” below). =”” this=”” test=”” has=”” some=”” problems=”” but=”” seems=”” to=”” be=”” the=”” best=”” available=”” at=””>
EFFECT .. GRP BY TIME
Multivariate Tests of Sig. (S = 1, M = 0, N = 22 1/2)
EFFECT .. TIME
Multivariate Tests of Sig. (S = 1, M = 0, N = 22 1/2)
Tests involving ‘TIME’ Within-Subject Effect
AVERAGED Tests of Significance for MEAS.1 using UNIQUE sums of squares
Note: These are the univariate tests of the main effect for TIME and the GROUP x TIME interaction which are subject to the sphericity assumption. Notice that, in this case, the MANOVA is a more powerful test than the univariate split-plot ANOVA. This is probably due to the sphericity assumption being violated. Usually, the univariate test is more powerful when its assumptions are met
The GLM syntax that produces similar output is:
Note: The WSFACTOR command specifies that TIME is the repeated measures factor with 3 levels. Then it specifies the type of contrasts used to form the segments from a variety of options (beyond the scope of this course). The one equivalent to the default in MANOVA is “DIFFERENCE”. The transformation matrix for the contrasts specified in this way is shown below (note pretest and posttest means are compared in the first contrast, then follow-up with the average of the pretest and the posttest in the second contrast). GLM automatically outputs the univariate tests of these contrasts as well as the tests of the overall main effects and interactions in the design that involve the within-subjects factor (or factors). In the “MEASURE” command, the segments are named (like the RENAME command in MANOVA). The TEST(MMATRIX) command is always included when a profile analysis is done. It is the SSCP matrix for testing the repeated measures effects using segments. The TEST (SSCP) outputs the matrices testing each multivariate effect and the pooled error SSCP matrix as before (not shown).
TIME
Measure: SEGMENT
Tests of Within-Subjects Contrasts
Measure: SEGMENT
The DESCRIPTIVES command gives the means in just the same way as MANOVA. As well, you can request partial eta2 and power for each effect by specifying these options on the /PRINT command. The SSCP, variance – covariance matrix, and correlation matrix for each between-subject condition are not part of the output, however. Rather the pooled matrices are produced (without the determinant!)
Residual SSCP Matrix
Note: The pooled correlation matrix for the dependent variables shows that the pretest and posttest correlate much more strongly with each other than with the follow-up (MANOVA only prints out the pooled variance – covariance matrix, but gives the determinant of this matrix as a direct index of multicollinearity).
The output showing the multivariate tests for two main effects and the interaction is identical to the MANOVA program. However, the univariate output contains the adjustments to the split-plot univariate ANOVA should sphericity be violated. Instead of using these corrections, however, it is better to use the multivariate results that do not rely on this assumption.
Tests of Within-Subjects Effects
Measure: SEGMENT
Note: Howell recommends the Huynth-Feldt correction and is more enthusiastic about using ANOVA and correcting for the sphericity assumption than TF.
Simple Main Effects Analysis and Contrasts Within These Effects
Researchers often do simple main effects with one dependent variable (ANOVA) to examine particular hypotheses (a priori tests), or explore the finding using post hoc simple main effects if the overall interaction is significant (as it is in this case). Logically, this analysis is not quite correct because the dependent variable in the profile analysis are segment scores (for effects involving the within-subjects factor or factors), whereas they are the original scores on the dependent variable in the simple effects analysis. The justification is that the researcher wants to interpret the pattern of findings in terms of the original scores (the segments were used to avoid the sphericity assumption). For example, it might be hypothesized that there is no difference between the groups at pretest (because of random assignment) and that the treatment has the most effect at posttest, so the simple main effects comparing the treatment group with the control group at pretest and posttest would be of interest. Also, the duration of the effect at follow-up could also be examined in a post hoc manner. These simple main effects are set up as contrast that are tested using the SPSS MANOVA program (see the following example). For post hoc analyses, a Scheffe adjustment to the F test for the contrast is necessary:
Fs = (k – 1) F [ k – 1, k (n – 1)]
Where k is the number of groups and n is the number of respondents within each group.
The MANOVA program is better than GLM for computing simple main effects. Consider the evaluation of a leadership training program. At the pretest, researchers hope that the training group and the control group are roughly equivalent. If the leadership training program is effective, however, the treatment group should have superior leadership skill than the control group at posttest and follow-up. Provided the interaction in the main analysis is significant, these simple main effects can be tested as follows:
If you run this analysis, the results show that the difference between the two groups is not significant at pretest, but is highly significant at posttest (p < .0001). =”” at=”” follow-up,=”” the=”” difference=”” is=”” only=”” marginally=”” significant=”” (p=””><> Another way to explore the results more fully if the interaction (parallelism) is significant, is to examine the differences among levels of the within-subjects factor for the treatment group and the control group separately. Here the full advantage of using MANOVA rather than GLM can be seen. The syntax is:
This analysis compares the pretest with the posttest followed by a second orthogonal contrast of little interest. The output directly tests the hypothesis that the posttest is greater than the pretest in the treatment group (a specific effect within a simple main effect).
The researcher can also test for these differences within the control group in a post hoc manner by specifying MWITHIN GROUP(2).
Some Clarifying Notes and Comments
Essentially, in ANOVA this statistic is equal to the SSeffect / SStotal and represents the proportion of variance in the dependent variable accounted for by the effect (e.G., the proportion of variance accounted for by a main effect). When this is used to infer (estimate) the effect size in the population from a sample, this statistic is an overestimate and it is better to use 2 – the one you were taught in Psy 805. In contrast, when estimating the effect sizes derived from a multivariate analysis of variance partial 2 is used. The SPSS GLM program also prints out partial 2 for the univariate tests, although it would be better to use 2 as the estimate for these univariate effect sizes.
The question of whether to use MANOVA or GLM to conduct a multivariate analysis of variance depends on the type of analysis that is needed and the type of output that is desired. For fully factorial designs, they are both very good. As TF point out, MANOVA is the more versatile and powerful program because 1) the determinant of the pooled variance – covariance matrix is part of the output, 2) the simple main effects analysis is more powerful and uses the pooled error term from the whole design, 3) it is the only program that allows the pooling of interaction terms with the covariate to test the homogeneity of regression assumption necessary before running a multivariate analysis of covariance, 4) it is the only program that provides the option of Roy-Bargmann’s stepdown analysis, 5) it is the only program that allows the researcher to conduct a PCA on the pooled correlation matrix specifying the relationships among the dependent variables, and 5) it is easier to use the syntax when specifying special contrasts. However, the much more modern GLM program can be run through windows and the results are displayed in a much clearer format. As well, GLM provides 2 , power estimates, and adjusted means for MANCOVA more easily. It is, however, worth the effort if you plan on regularly using multivariate analysis of variance to analyse your data. For applied researchers who evaluate programs using randomized and quasi-experimental designs, this is a must program to know!