T-tests

The Good News About T-tests

We’ve arrived! Sort of! You’ve learned how to describe your data with measures of central tendency and dispersion (after you’ve determined the appropriate level of measurement), you’ve learned about probability and why we care about normal distributions and how we can know that the mean of any sample distribution is likely pretty close (within 1 standard deviation) of the population mean. You’ve also learned how to tell if a sample mean differs significantly from a population mean if we have the population mean and standard deviation.

With these tools, you can already answer some interesting questions! But, many of our questions are not about differences between a sample and the population, but differences between groups. I think of criminals and non-criminals, men and women, people with certain risk levels or number of victimizations or a pre-test-post-test experimental design. How do we answer those questions?

Well, really, it depends! But we can begin to answer them with t-tests! This is the first really big gun in your arsenal because t-tests allow us to test group mean differences between two groups.

More Good News!

In addition to providing you the ability to test between two groups, t-tests are basically generalized z-tests! So the steps in the process are exactly the same.

1. Formulate your hypotheses
2. Set \(\alpha\)
3. State your decition rule
4. Define the critical value
5. Calculate the t-test
6. State your results and conclusions

Let’s Do This!

First let’s get some data in. I’m going to use our class’s normal NYFS dataset for this. In particular, I have a question about the rates of crime between men and women in that data. I’ll provide the first part of the data, below.

As with z-tests (and really all hypothesis testing), the first thing I need to do is create my hypotheses.

H₀: \(\mu_p = \mu_s\) is the null hypothesis

H₁: \(\mu_p \neq \mu_s\) is the research hypothesis

In this particular case, I’m interested in difference between men and women in relation to crime. Specifically, I think there is a difference between men and women when it comes to minor offending. Let’s reformulate our hypotheses with more appropriate formulas, since we’re not using populations anymore, and we’re using two groups from our sample rather than a population and a sample.

H₀: \(\bar{x}_m = \bar{x}_w\) is the null hypothesis

H₁: \(\bar{x}_m \neq \bar{x}_w\) is the research hypothesis

We have to keep in mind a couple of things. Now, I’m talking in terms of group means. This is important because given values may be quite different within groups. So, my null hypothesis is that the mean of men’s minor offending is equal to the mean of women’s minor offending. Our (non-directional) research hypothesis is that the mean of men’s minor offending is different from the mean of women’s minor offending.

As (just about) always, our \(\alpha\) is .05.

Combining Some Things

If you look at our list above, now we’d normally state the decision rule, find our critical value, and then calculate the test. The reality is, we still do these things, but we essentially combine them into a single step. We can do this by calculating the t-test itself. The test gives us the test statistic (in this case \(t\)), and an associated p-value. From these, we can know if \(t\) fits our decision rule and if it exceeds our critical value by whether \(p < .05\).

## 
##  Welch Two Sample t-test
## 
## data:  nyfs$minor by nyfs$sex
## t = -1.8147, df = 530.29, p-value = 0.07013
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
##  -10.4570961   0.4143238
## sample estimates:
## mean in group female   mean in group male 
##             4.625874             9.647260

We can see from the above test that there is not a statistically significant difference between the two groups as our p-value is greater than .05 (\(p = .07013\)). Given the difference in means, this seesm somewhat surprising. But let’s visualize it to see if we can figure out why.

You can see in the image above that the distributions are incredibly hard to distinguish. Additionally, the lines are quite close together, given the range of the distribution! So, even though we have quite a few people in our sample, and even though there is a reasonable distance between the two means, when we take into account the full distributions, it’s easy to see why we can’t be sure this difference is real.

So What’s Going On?

Like a z-test, a t-test is interested in differences between means, but importantly, the t-test takes the distribution of both groups into account! The z-test, if you remember, only requires the standard deviation of the population, it effectively ignores this statistic for the sample. Why? Because if we know that the population is distributed normally, and we know that when we have a sufficient enough sample size (40ish) that the mean of the sample will fall reasonably close (within 2 standard deviations) to the population mean, the standard deviation for the sample doesn’t matter. But when you’re dealing with two groups from a sample, all of a sudden we need to care! Now we’re not talking about a theoretical normal distribution with a known mean and standard deviation (as we were with z-tests), now we have two real, empirical distributions and we can take that information into account when we’re looking for the difference.

Another Example

Let’s take another example and run through it. This time, we’ll use race (ethnic2) as the grouping variable. Let’s look at the difference in marijuana use between whites and other races.

First, we need to formulate our null and research hypotheses. I’m going to hypothesize that whites use more marijuana than others.

H₀: \(\bar{x}_w <= \bar{x}_o\) is the null hypothesis

H₁: \(\bar{x}_w > \bar{x}_o\) is the research hypothesis

\(\alpha\), as always, is set to .05.

## Loading required package: carData

Table 1: Welch Two Sample t-test with alternative hypothesis: \(\mu_1 < \mu_2\)

Diff	\(\mu_1\) Other	\(\mu_2\) White	t	SE	df	\(CI_{95\%}\)	p	Cohen's d
12.8	33.15	20.35	1.01	12.69	123.41	(-Inf - 33.82)	.842	0.15

As we can see, the p-value is .842, much higher than the .05 we need it to be less than. Based on this, we can say that about 84% of the time we’d be likely to get this answer by chance. Importantly, this is about twice the p-value we would have if the test were non directional.¹

Again, visualization helps.

Given thsoe distributions, we can see that the means are not far enough apart to say that the difference is not by chance.

Some Things to Keep In Mind

So, t-tests are useful! And they function basically like z-tests. But there are a couple of important things to remember.

1. T-tests only work with two groups
2. Group variances have to be equal (within reason - close is good enough)
3. You need 1 categorical variable (factor) and one continuous variable
4. The groups cannot be related

The last point is one that is easy to miss. What we mean by unrelated is that a case cannot be a member of both groups. So, in other words, if it’s a pre-test/post-test situation, where you test the same people twice, you can’t use regular t-tests. There is a different version for that called paired t-tests.

Unequal Variances and Paired t-tests

Unequal variances

So what happens if you have unequal variances between your groups? And how do you know? Fortunaetly, R has your back. The specific test we are using, the Welch Two Sample t-test, corrects for unequal variances by default. We can set the parameter var.equal to TRUE if we want to assume we have equal variances. The reality is that, in the vast majority of cases, the difference between unequal and equal variances assumed in the t-test doesn’t make a huge difference. That said, what’s happening here is worth visualizing using something called a quantile-quantile plot. We can use gender and alcohol as the variables. I have also limited it to those who used under 50 times in the year because that data is much less spread out than the full distribution.

Here we can see that while there are not a ton of cases directly on the diagonal, they follow the same general pattern. This suggests that the variances are, while not quite equal, fairly close. If everything fell directly on the diagonal, it would be perfectly equal variances and if nothing was on the diagonal it would be perfectly unequal. In the above case, it’s better to use unequal variances (the default) to be safe.

Paired-sample t-tests

So what do you do if you have data where people can belong to both groups? Well, fortunately, it’s easy to deal with this as well using a paired-sample (or dependent sample) t-test. It functions exactly like a regular t-test, but has a correction for the correlated errors we see when people belong to both groups. We use this type of data when we have experimental conditions available.

ID<-c(1:12) # create case IDs
before<-c(5,4,6,6,7,9,9,2,3,4,5,5) # before scores
after<-c(5,4,5,5,6,7,7,2,5,4,4,5)# after scores
data<-data.frame(ID, before, after) # create a dataframe from the above variables
head(data) # show the first 6 rows

In the above chunk, I have created a dataframe from some vectors I created that represent our before and after tests. We’ll assume here that the before scores represent the crime severity scores for 12 people in the population and the after scores represent the same people’s crime severity after I’ve given them a treatment. We can’t use an independent-samples t-test because people are in both the before and after groups! Instead, we use a paired sample t-test. Normally, we would go through all 6 steps as we did above, but to save time and space, we’ll just say that we expect them to be different.

## 
##  Paired t-test
## 
## data:  data$before and data$after
## t = 1.5933, df = 11, p-value = 0.1394
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
##  -0.1907197  1.1907197
## sample estimates:
## mean of the differences 
##                     0.5

Sad day! It looks like my treatment was not successful…

So What?

So now you know how to do t-tests! They’re very useful and serve as the basis for quite a few other tests. But, as you might imagine, they’re not quite enough by themselves. Later, we’ll learn what to do when you have more than two groups and when you’re interested in relationships rather than differences.

Helpful Video

---

1. Because directional hypotheses use only one tail, they represent more conservative tests.↩