T-Test and F-Test: Fundamentals of Test Statistics

Statistics is all about coming up with models to explain what is going on in the world. But how good are we at that? I mean, numbers are only good for so many things, right? How do we know if they are telling the right story?

Enter the famous world of test statistics.

The goal of a test statistic is to determine how well the model fits the data. Think of it a little like clothing. When you are in the store, the mannequin tells you how the clothes are supposed to look (the theoretical model). When you get home, you test them out and see how they actually look (the data-based model). The test-statistic tells you if the difference between them (because I definitely do not look like the mannequin.) is significant.

In another post, I discussed the nature of correlational and experimental research. Linear regression, multiple regression, and logistic regression are all types of linear models that correlate variables that occur simultaneously. However, experimental models are concerned with cause-effect models, or at least models that state a significant difference between cases.

Test statistics calculate whether there is a significant difference between groups. Most often, test statistics are used to see if the model that you come up with is different from the ideal model of the population. For example, do the clothes look significantly different on the mannequin than they do on you? Let’s take a look at the two most common types of test statistics: t-test and F-test.

t-Test and Comparing Means The t-test is a test statistic that compares the means of two different groups. There are a bunch of cases in which you may want to compare group performance such as test scores, clinical trials, or even how happy different types of people are in different places. Of course, different types of groups and setups call for different types of tests. The type of t-test that you may need depends on the type of sample that you have.If your two groups are the same size and you are taking a sort of before-and-after experiment, then you will conduct what is called a Dependent or Paired Sample t-test. If the two groups are different sizes or you are comparing two separate event means, then you conduct a Independent Sample t-test.

Dependent or Paired Sample t-Test

I am a fairly introverted person. I’m so introverted that I have extreme anxiety in social situations that warrant a therapy dog by the name of Chloe. And she’s pretty adorable.

Now, a lot of people have therapy dogs in order to relieve anxiety. Let’s say that you measure people’s anxiety without their therapy dogs and with their therapy dogs on a scale from 1 (low) to 5 (high) to determine if therapy dogs do significantly lower anxiety for people like me. For the sake of convenience, you get the following data

At first glance, it seems that there is a clear difference between people’s level of anxiety with and without their therapy dogs. You want to jump to the conclusion that our model (they do make a difference) is different from the null hypothesis (they don’t). But wait, you want to have some statistical data to back that claim up. So you perform a t-test.

A t-test is a form of statistical analysis that compares the measured mean to the population mean, or a baseline mean, in terms of standard deviation. Since we are dealing with the same group of people in a before-and-after kind of situation, you want to conduct a dependent t-test. You can think of the without scenario as a baseline to the with scenario.

The traditional t-test equation looks like

The null hypothesis states there should be no difference between the two sample means. So that means μ1 – μ2 = 0 giving us

But what do you do with this number? Well, you will consult the mystical chart of t Table. Along the top of the table is the probability of error that you are willing to accept. In other words, what is the possibility that you are wrong? Along the side of the table are the degrees of freedom. In this case, you have 46 degrees of freedom because you have two groups with 24 participants each.

The t Table states that the critical value for 46 degrees of freedom and the 0.05% error is 2.013. Your calculated t-value is above that, which indicates that your means are significantly different. Based on my completely random, fictitious data, the lower mean of anxiety people show with their therapy dogs is different enough to be meaningful, otherwise known as statistically significant.

I guess Chloe is good for me, lol.

Independent Sample t-Test

The case for independent sample tests is a little different. This style of test is best suited to experimental designs, or those designs that compare groups with different sets of participants. The benefit is that the groups do not have to be equal sizes. Let’s check another statistical example.

Let’s pretend for a moment that you (for some crazy reason) want to know if people are more anxious in statistics class than in another, let’s say English, class. So you find some willing volunteers and measure their heart rates during each class. It’s important to note that neither class will have the same participants. Your data looks a little like this

There is a difference, but is it enough of a difference? When you calculate the t-value and find it to be 1.92, compare this to the t-table at the 40 mark, notice it is below the critical value. This means that while there is a difference, it is not a significant difference.

Huh, I guess statistics isn’t too stressful after all.

The role of the t-test is to determine whether two groups are different from each other. Just remember that dependent t-tests are best used for groups that have the same participants, while independent t-tests are for groups with different ones.