ANOVA Theory
ANOVA Theory
Dr. Hill
Why ANOVA?
Imagine a situation in which you want to compare men and women on some continuous variable - say, height. You have access to a reasonable-sized group of people (say, 1000), with roughly equal numbers of men and women. To test this, you would simply measure everyone’s height, record it along with whether they are a man or woman (I’m assuming two genders for the sake of ease, here), and use a t-test to compare the means of the two groups. Basically this:
\(\mu_1 = \mu_2\)
But what happens when you have three groups? Couldn’t we just do a set of t-tests?
Let’s take an example: Now we have a group of people who score “high”, “medium”, and “low” on an agressiveness scale, and we want to see if that affects whether the speed while driving. We have access, as in the above example, to a sufficiently sized sample, and we can measure them on both their agressiveness (and classify them into the categories above) and their speed while driving. So, to test whether or not there is a difference between groups, we could, in principle, conduct the following tests:
\(\mu_1 = \mu_2\), \(\mu_1 = \mu_3\), and \(\mu_2 = \mu_3\)
But we have a couple of issues: First, we know that (roughly) 5% of the time we are willing to accept an incorrect answer (that’s why \(\alpha\) is set at .05), but now we are doing multiple tests, compounding the likelihood that we will get an answer by chance! Also, while we can adjust for this (using something called the Bonferoni adjustment) multiple testing, imagine if we had more than 3 groups! With only 4 groups, we’d have:
\(\mu_1 = \mu_2\), \(\mu_1 = \mu_3\), \(\mu_1 = \mu_4\), \(\mu_2 = \mu_3\), \(\mu_2 = \mu_4\), and \(\mu_3 = \mu_4\) - that’s 6 tests!
ANOVA gives us a statistic - \(F\) - that allows us to calculate overall difference between groups. This is a single statistical tests - meaning no adjustment - rolling everything (at least initially) into a single test.
ANOVA in Principle
In principle, One-Way ANOVA (as opposed to factorial ANOVA) is very straightforward. Essentially, we want to see how the variation is between groups compared to the variation within groups. In fact, this is literally the calculation we use!
\[F = \frac{MS_{between}}{MS_{within}}\]
Where \(MS_{between}\) is the variance between groups and \(MS_{within}\) is the variance within groups.
What this means is that \(F\) is a ratio! This makes is super easy to understand. Let’s run through an example.
Easy Conceptual Example
Imagine you have the same group of drivers we mentioned above, and we have the appropriate measures of their speed and agressiveness. Now, imagine if every driver within each group had the same recorded speed. In other words, all the drivers with low agressivness drove exactly the same speed. All the drivers with medium agressivness drove the same speed and the same for those who scored high. The difference between the groups would then be really easy to calculate by simply comparing each speed to the others.
In this example, we have all the “high” aggressivness people going 75mph, all the “medium” going 70 and all the “low” going 65. Thus, we could simply do subtraction (\(High - Medium - Low\)) to find out the differences.
Alas. Reality is often much more complicated, and we know that we are likely to have a range of values within each group, as well as between groups. This means that, even though we could still calculate a mean for each group and subtract, we wouldn’t be taking into account the dispersion within each group. And, as we recall from earlier, the dispersion really matters because it helps us understand how our distribution looks.
Let’s take the example from above and flesh it out:
I re-created the dataset, but this time rather than single values, each group has a distribution of values. Given how spread out each group is, it’s now much more difficult to see if the difference is real. Another way to visualize this idea is below.
> The means for each group are represented by the vertical solid lines.
It’s pretty tough to pull these groups apart, even if the means seem distinct!
Why? Because even though the means are pretty far apart (at least when comparing high and low groups), the spread (AKA variance) of the distributions makes it tough to know if they’re really different or not. Fortunately, that’s exactly what ANOVA does.
So, back to the ratio idea:
If the variance between groups and within groups are the same, the ratio is equal to 1. That is to say, there is no difference in between-group or within-group variation. In other words, each group’s distribution would look exactly the same as the overall distribution - we would essentially have only 1 distribution. If, however, the between-group value gets larger, without the within-group value also getting larger, we can be more and more certain that the between group differences are real. In short, as \(F\) gets bigger, the difference between groups (while accounting for within-group variation) is larger.
Degrees of Freedom
When we discussed t-tests, we also used degrees of freedom (and you should use them in your reporting). I didn’t spend much time on them, outside of sayint that they’re essentially representing the amount of informtion we can use. In t-tests, degrees of freedom (\(df\)) are calculated \(n_1 - 1 + n_2 - 1\) where \(_n\) is the size of each of the groups.
ANOVA also requires us to use (and report) \(df\), but the calculation is different. We need \(df\) for both the numerator (\(MS_{between}\)) and denominator ($MS_{within}). Fortunately, these are easy to calculate, and (as we’ll see in the application portion) R does it for us. For completeness however, the formula for the numerator is:
\[ df_{MS_{between}} = k - 1\]
Where k is the number of groups. The denominator is slightly different:
\[ df_{MS_{within}} = N - k\]
Where \(N\) is the number of cases. These values are used to determine the critical value of \(F\), which, were we to calulate by hand, we would look up using these values along with the \(\alpha\) value of .05 we set earlier to determine if we met our decision criteria.
Comments
Post a Comment