Factorial ANOVA
Factorial ANOVA
Adding Variables to ANOVA - Factorial ANOVA
While One-way Analysis of Variance is a powerful tool, it’s rare within social science (and practical matters) that we’re interested only in the dfferences across a single dimension (e.g. differences in only race). More often, we have multiple dimensions that we’re interested in testing, and frequently, we’re interested not just in what we would term their direct effects, but also how those variables interact.
The good news is that ANOVA is very flexible, and can handle multiple variables. While it can (technically) handle all kinds of levels of measurement, the version of ANOVA we’re going to work with in this lesson is Factorial ANOVA. That is, ANOVA by more than one factor. This both helps to simplify the explanation (if we used interval/ratio level data, we’d be moving into something caled ANCOVA), but it also will help us understand regression a bit better when we get to it a bit later in the semester (like, next week).
Speaking of regression, it’s good to keep in mind how Factorial ANOVA is written in R because regression is essentially written exactly the same way. The simple reason is, as we’ll see, they’re doing very similar things (and even using some of the same statistics, like \(F\)).
Thinking About What We’re Doing
So, let’s think about how we’re approaching this. In One-way ANOVA, we’re interested in the difference in means between a set of groups. In Factorial ANOVA, we’re interested in the same thing, but the groups are defined across two dimensions (whatever your factors are). By way of example, let’s use a couple of tables:
| Male | Female | |
|---|---|---|
| Means | 23 | 28 |
In the table above, we have the (hypothethical) mean of two groups, males and females. We could use a t-test to examine the difference between these two groups, but the same idea applies to One-way ANOVA; we’re testing means along a single dimension - gender.1
In the case of Factorial ANOVA, we’re expanding the number of dimensions.
| Other | White | |
|---|---|---|
| Female | 58 | 50 |
| Male | 68 | 69 |
In the table above, we’ve moved from a 1x2 table to a 2x2 table. Each one of the cells represents a mean for us to compare. What we mean by a Factorial ANOVA is that we are factoring the ANOVA by more than one variable.
Main Effects vs. Interaction Effects
In the first table above (the 1x2), we were interested in the difference between males and females (in terms of their group means). The cool part of Factorial ANOVA is that we can still answer this question! But we can add a couple of additional questions as well. In fact, we can answer 3 questions:
- Is there a difference caused by gender on the score?
- Is there a difference caused by race on the score?
- Is there a difference caused by race and gender combined on the score?
The answers to the first two questions we call main effects. The answer to the last question we call an interaction effect. In principle, you could just two two ANOVAs to answer the first two questions (though you’d have to adjust the p-value for multiple tests). However, the last question cannot be answered by another One-way ANOVA. You have to use Factorial ANOVA to find out the answer.
Diving In
The best way to understand Factorial ANOVA is by running through an example. In this case, I’m going to be examining gender and ethnicity’s impact on minor offending. To simplify, I’m going to recode ethnicity from the original data (which had 4 groups) into 2 groups, “White” and “Other.”2
Normal Steps
As usual, it’s best to start with a visualization. In this case, it can be helpful to visualize the groups independently first, and then use a new (to us) type of visualization called an interaction plot.
In the above plots, we can get some insight into our first two questions: Is there an impact of Sex or an impact of Race on minor offending? We can see that it doesn’t look like there’s a whole of of difference between the sexs or races in terms of minor offending, suggesting that we shouldn’t expect to find significant differences. This visualization can be informative later, when we’re interpreting our results.
However, while the above helps us get some initial information on our main effects (gender on minor offending and race on minor offending), it is not informative when it comes to the interaction effect we mentioned in the above section. Fortunately, R provides us with a type of plot called an interaction plot, to help us visualize whether we should see any interaction effects.
interaction.plot(nyfs$sex, nyfs$ethnic2, nyfs$minor)
Interpreting the above plot is fairly straightforward. On the x-axis we have one of our factors, and the line-types represent our other. The y-axis represents the mean of minor offenses for each group. In both cases, we see the means of males is higher than the means of females. However, you can see that the lines cross. This is usually indicative of some type of interaction, is it suggests that the combination of variables (gender and race) operate differently across groups. However, this does not tell us whether it will be significant.
Hypotheses & \(\alpha\)
As with all statistical tests, we need to set up our hypotheses. In terms of Factorial ANOVA, this is largely similar to what you’ve been doing with t-tests and F-tests before. However, there is one difference, which is that interaction effects have their own distinct hypothesis (and null). All of our hypotheses for the current example are set up, below.
H0A: \(\bar{X}_{males} = \bar{X}_{females}\) is the null hypothesis for gender
H1A: \(\bar{X}_{males} \neq \bar{X}_{females}\) is the research hypothesis for gender
H0B: \(\bar{X}_{whites} = \bar{X}_{others}\) is the null hypothesis for race
H1B: \(\bar{X}_{whites} \neq \bar{X}_{others}\) is the research hypothesis for gender
H0C: \(\bar{X}_{white-male} = \bar{X}_{white-female} = \bar{X}_{other-male} = \bar{X}_{other-female}\) is the null hypothesis for the interaction effect
H1C: \(\bar{X}_{white-male} \neq \bar{X}_{white-female} \neq \bar{X}_{other-male} \neq \bar{X}_{other-female}\) is the research hypothesis for the interaction effect
If you look through the above hypotheses, you can see that understanding what we’re` testing hasn’t really changed. We’re still just interested in finding the difference between group means. But how we define the groups has changed because we have more dimensions (read:variables) to factor them by. In this case, with our interactions, we can see that there are 4 groups between whom we are testing differences: white males, white females, other males, and other females.
\(\alpha\), as always, is set to .05.
Conducting the Factorial ANOVA
In the code produced below, you can see the development of the model is just a logical extension of the One-Way ANOVA function. As I mentioned above, it’s important to get used to this notation, as we both Factorial ANOVA and linear regression (covered next week) are based on what’s called the General Linear Model (GLM), and so it’s worth understanding the notation.
You should notice that there are a couple of differences between One-way and Factorial ANOVA formulas. The first is that we literally add factors by using a \(+\). The second is that we introduce interaction effects with a * operator. That’s just a way for you to tell R which interactions you’re interested in (as later with regression we can include virtually unlimited interactions.)
model1<-aov(nyfs$minor ~ nyfs$sex + nyfs$ethnic2 + nyfs$sex*nyfs$ethnic2)
model1$call## aov(formula = nyfs$minor ~ nyfs$sex + nyfs$ethnic2 + nyfs$sex *
## nyfs$ethnic2)summary(model1)## Df Sum Sq Mean Sq F value Pr(>F)
## nyfs$sex 1 3643 3643 3.262 0.0714 .
## nyfs$ethnic2 1 172 172 0.154 0.6952
## nyfs$sex:nyfs$ethnic2 1 220 220 0.197 0.6576
## Residuals 574 640974 1117
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1Looking at the model we see above (we get results of the object we saved with the summary() function), as with One-way ANOVA, we’re most interested in the p-values associated with each of our variables (Pr(>F)). As none of the values are less than .05, we can see that there are no statsitically significant differences across our groups.
This means, looking back at our hypotheses, we cannot reject any of the null hypotheses.
Breif Digression
It’s worth thinking through that formula we put into the aov() function a bit more deeply. Here it is in a generalized form:
\[dependent = \alpha + \beta(independent_1) + \beta(independent_2) + ...+\beta(independent_k) + e\]
While the above looks complex, keep in mind what it’s actually saying: The dependent variable is a function of the slopes (\(\beta\)) of the independent variables plus error. In the case, the function is additive (e.g. the effects of one independent variable are added to the others) and linear (we are not using curves - we’re using lines). \(\alpha\) is just the y-intercept (the average value of the dependent variable if all the independent variables were 0) and \(e\) is the error term - which we can’t do anything about and so we ignore it.
We’ll be going over this again in more detail with regression, but keep in mind, in its simplist form, this is just:
\[y=m(x) + b\]
If you don’t remember this formula from high school, don’t worry. You really only have to remember one thing from this digression:
All we’re really doing is calculating the slope of a line and then seeing what effects other variables have on the slope (and whether they’re significant).
As I said, we’ll go over this again with regression, but it’s important to see that ANOVA and regression really are not doing different things.
An Alternative Example
But what if we did find significance in either our main effects or our interaction effect? Let’s try another example and see what we find. In this case, I’m going to use a more complex factor: ethnicity without a recode. I’ll keep gender as my second factor. Additionally, we’ll look at alcohol use as the dependent variable.
nyfs$ethnic<-as.factor(nyfs$ethnic)
par(mfrow=c(1,2))
plot(nyfs$ethnic, nyfs$alcohol, xlab="Ethnicity", ylab="Number of Drinks in the Past Year")
plot(nyfs$sex, nyfs$alcohol, xlab="Sex", ylab="Number of Drinks in the Past Year")
Already we can see that there are likely some differences when looking at the factors independently. We can also check the interactions.
interaction.plot(nyfs$sex, nyfs$ethnic, nyfs$alcohol, xlab="Sex", ylab="Mean of Group Alcohol Consumption")
Examining the above interaction plot, we can see that there appear to be differential effects on males and females depending on what ethnic group they’re in. This, again, can give us some clues later when we want to interpret our results.3
Running the ANOVA is simple.
model2<-aov(nyfs$alcohol ~ nyfs$ethnic + nyfs$sex + nyfs$ethnic*nyfs$sex)
summary(model2)## Df Sum Sq Mean Sq F value Pr(>F)
## nyfs$ethnic 3 53146 17715 3.377 0.01815 *
## nyfs$sex 1 46119 46119 8.790 0.00315 **
## nyfs$ethnic:nyfs$sex 3 12132 4044 0.771 0.51067
## Residuals 570 2990516 5247
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1Looking at the above table, we can see that two of our effects are significant. Examining the rows for ethnic and for sex, we can see that the p-values are \(< .05\), indicating that there are significant differences between the group means. We can also see that, despite our interaction chart suggesting otherwise, we do not see an interaction effect (\(p > .05\)).
Post-hoc Tests and Factorial ANOVA
As with One-way ANOVA, we need to conduct post-hoc tests to understand which differences are significant if and only if we have significant results in our main model. We do this in the same way as One-Way ANOVA:
TukeyHSD(model2)## Tukey multiple comparisons of means
## 95% family-wise confidence level
##
## Fit: aov(formula = nyfs$alcohol ~ nyfs$ethnic + nyfs$sex + nyfs$ethnic * nyfs$sex)
##
## $`nyfs$ethnic`
## diff lwr upr p adj
## American Indian-African American 1.165437 -55.017319 57.34819 0.9999451
## Hispanic-African American 35.008095 -10.383554 80.39974 0.1939790
## White Non-Hispanic-African American 26.323375 2.844544 49.80221 0.0208746
## Hispanic-American Indian 33.842657 -31.446028 99.13134 0.5405777
## White Non-Hispanic-American Indian 25.157938 -27.315544 77.63142 0.6045944
## White Non-Hispanic-Hispanic -8.684720 -49.395479 32.02604 0.9466269
##
## $`nyfs$sex`
## diff lwr upr p adj
## male-female 17.84811 6.012341 29.68387 0.0031851
##
## $`nyfs$ethnic:nyfs$sex`
## diff lwr
## American Indian:female-African American:female -10.833333 -107.4660650
## Hispanic:female-African American:female 50.333333 -31.1545409
## White Non-Hispanic:female-African American:female 18.481322 -19.6546025
## African American:male-African American:female 6.196078 -45.5069041
## American Indian:male-African American:female 16.809524 -73.6435897
## Hispanic:male-African American:female 29.282051 -41.2885179
## White Non-Hispanic:male-African American:female 39.666667 1.5999841
## Hispanic:female-American Indian:female 61.166667 -54.9714229
## White Non-Hispanic:female-American Indian:female 29.314655 -61.8013749
## African American:male-American Indian:female 17.029412 -80.5461006
## American Indian:male-American Indian:female 27.642857 -94.9521997
## Hispanic:male-American Indian:female 40.115385 -68.6411667
## White Non-Hispanic:male-American Indian:female 50.500000 -40.5870711
## White Non-Hispanic:female-Hispanic:female -31.852011 -106.7153521
## African American:male-Hispanic:female -44.137255 -126.7409439
## American Indian:male-Hispanic:female -33.523810 -144.5730634
## Hispanic:male-Hispanic:female -21.051282 -116.6042845
## White Non-Hispanic:male-Hispanic:female -10.666667 -85.4947587
## African American:male-White Non-Hispanic:female -12.285243 -52.7505929
## American Indian:male-White Non-Hispanic:female -1.671798 -86.2058850
## Hispanic:male-White Non-Hispanic:female 10.800729 -52.0041378
## White Non-Hispanic:male-White Non-Hispanic:female 21.185345 0.8551104
## American Indian:male-African American:male 10.613445 -80.8461716
## Hispanic:male-African American:male 23.085973 -48.7701405
## White Non-Hispanic:male-African American:male 33.470588 -6.9295121
## Hispanic:male-American Indian:male 12.472527 -90.8322080
## White Non-Hispanic:male-American Indian:male 22.857143 -61.6457296
## White Non-Hispanic:male-Hispanic:male 10.384615 -52.3782314
## upr p adj
## American Indian:female-African American:female 85.79940 0.9999742
## Hispanic:female-African American:female 131.82121 0.5659037
## White Non-Hispanic:female-African American:female 56.61725 0.8209637
## African American:male-African American:female 57.89906 0.9999594
## American Indian:male-African American:female 107.26264 0.9992360
## Hispanic:male-African American:female 99.85262 0.9122291
## White Non-Hispanic:male-African American:female 77.73335 0.0341958
## Hispanic:female-American Indian:female 177.30476 0.7489504
## White Non-Hispanic:female-American Indian:female 120.43069 0.9773041
## African American:male-American Indian:female 114.60492 0.9994941
## American Indian:male-American Indian:female 150.23791 0.9973461
## Hispanic:male-American Indian:female 148.87194 0.9518873
## White Non-Hispanic:male-American Indian:female 141.58707 0.6960141
## White Non-Hispanic:female-Hispanic:female 43.01133 0.9008673
## African American:male-Hispanic:female 38.46643 0.7347172
## American Indian:male-Hispanic:female 77.52544 0.9842662
## Hispanic:male-Hispanic:female 74.50172 0.9977093
## White Non-Hispanic:male-Hispanic:female 64.16143 0.9998687
## African American:male-White Non-Hispanic:female 28.18011 0.9837382
## American Indian:male-White Non-Hispanic:female 82.86229 1.0000000
## Hispanic:male-White Non-Hispanic:female 73.60560 0.9995410
## White Non-Hispanic:male-White Non-Hispanic:female 41.51558 0.0341863
## American Indian:male-African American:male 102.07306 0.9999674
## Hispanic:male-African American:male 94.94209 0.9774831
## White Non-Hispanic:male-African American:male 73.87069 0.1888768
## Hispanic:male-American Indian:male 115.77726 0.9999573
## White Non-Hispanic:male-American Indian:male 107.36002 0.9918050
## White Non-Hispanic:male-Hispanic:male 73.14746 0.9996445While the above output may appear confusing, it is the same as the output we got from the TukeyHSD() function in the One-Way ANOVA, though now it is broken down into 3 sets of output: the two main effects and the interaction effect. We know that the two main effects were significant, so we need to examine where those differences lie, but we can ignore the interaction effects since it was not significant in the original model.4
Looking at the “p adj” column for race, we can see that White Non-Hispanics and African Americans differ (\(p < .05\)) in terms of their alcohol use, with White Non-Hispanics using significantly more alcohol. No other ethnic group comparisons were significant. Examining sex, we can see that there was a difference between men and women (\(p < .05\)), with men using statistically more alcohol than women on average.5
Effects Size
While \(\eta^2\) is the effects size of choice for One-way ANOVAs, we use \(\omega^2\) for the effects size of Factorial ANOVA. In the code below, I provide two ways of getting \(\omega^2\). The first is by defining a new function, the second is through a library called sjstats. Both are included in the code, below.
# If you run the code (between the curly braces - {}), it will create a new fucntion called omega_sq(). Then you can just use it with the model you created (as I do below).
omega_sq <- function(aov_in, neg2zero=T){
aovtab <- summary(aov_in)[[1]]
n_terms <- length(aovtab[["Sum Sq"]]) - 1
output <- rep(-1, n_terms)
SSr <- aovtab[["Sum Sq"]][n_terms + 1]
MSr <- aovtab[["Mean Sq"]][n_terms + 1]
SSt <- sum(aovtab[["Sum Sq"]])
for(i in 1:n_terms){
SSm <- aovtab[["Sum Sq"]][i]
DFm <- aovtab[["Df"]][i]
output[i] <- (SSm-DFm*MSr)/(SSt+MSr)
if(neg2zero & output[i] < 0){output[i] <- 0}
}
names(output) <- rownames(aovtab)[1:n_terms]
return(output)
}
# Using the newly defined function
omega_sq(model2)## nyfs$ethnic nyfs$sex nyfs$ethnic:nyfs$sex
## 0.01203871 0.01315425 0.00000000# Or use another library.
#install.packages("sjstats")
library(sjstats)## Registered S3 methods overwritten by 'lme4':
## method from
## cooks.distance.influence.merMod car
## influence.merMod car
## dfbeta.influence.merMod car
## dfbetas.influence.merMod car##
## Attaching package: 'sjstats'## The following object is masked _by_ '.GlobalEnv':
##
## omega_sqanova_stats(model2)As you can see, though the differences are real, they are not particlarly important. Ethnicity only explains about 1.2% of the variance in alcohol consumption while sex explains about 1.3%. In both cases, these would be considered weak effects sizes.
Conclusion
While Factorial ANOVA is slightly more complicated than One-way ANOVA, it gives us the ability to do far more. By allowing us to define both main and interaction effects, we can see not only the independent effects of each of our factors, but what impacts they have when combined on a dependent variable. Additionally, the model underlying ANOVA is the same as the model underlying regression and other types of linear analysis. As we’ll see, this suggests that while ANOVA is frequently a useful tool in the box, we can use (and understand) regression in many of the same ways.
Ethnicity with 4 attributes would be a good example with more categories - one we might use in a One-way ANOVA↩
I’m recoding the ethnic variable to two groups, but Factorial ANOVA works with any number of groups. Try it with the unrecoded ethnic variable and see what happens!↩
For sake of space and time, I’m going to assume you know how to set up the hypotheses and null hypotheses since it’s just a logical expansion on the earlier example.↩
If you look at the interaction effect p-values for signicicance (which you should never do if the results are not significant), you’d see that the only significant difference is between White males and females, which is subsumed completely into the difference between men and women in the main effects. That’s why the interaction was not significant.↩
While you can visualize these with plot(TukeyHSD(model2)), getting the plots into an interpretable format is not trivial.↩
Comments
Post a Comment