Chapter 6: t test for dependent samples ****This chapter corresponds to chapter 11 of your book (“t(ea) for Two (Again)”).
What it is: The t test for dependent samples is used to determine whether the means of two related groups are significantly different. It is called a t test for dependent samples because you use it when you are comparing the same group of people that was measured twice (i.e., groups that are related to or dependent on each other). When to use it: Per the flow chart on page 190 of Salkind (2008), you would use the t test for dependent samples when: (1) you are examining differences between groups (as opposed to examining the relationship between two variables) and (2) the participants in the study were tested more than one once (as opposed to once) and (3) You are comparing two groups/time points (as opposed to three or more groups/ time points) Questions asked by the t test for dependent samples: Do the means of two groups of related scores differ from each other? Examples of research questions that would use a dependent t test: o o o
Do the students in this class know more about SPSS at the beginning or end of the semester? Does people’s meaning in life increase after they have a child? Do people lose weight after attending a weight loss boot camp for 2 months?
Using SPSS to Calculate a dependent t test (Data set: Chapter6_Example1.sav) Imagine you’ve designed a method that you believe can teach algebra to people in a very short amount of time (15 minutes). To test this method, you recruit 10 random individuals and give them an algebra test of 10 problems. You then have them work through your method for 15 minutes and then they complete another algebra test of 10 problems. Now you want to compare the scores on the pre and post-test to determine if scores improved after participants went through your training. To determine whether your method worked or not, follow the famous 8 steps…. 1. Statement of the null and research hypotheses Null H0 : µpretest = µposttest The average score on the pretest and posttest are equal. Research H1: X pretest ≠ X posttest The average score on the pretest and posttest are not equal.
2. Setting the level of risk to p < .05 3. Selection of the appropriate test statistic Again, using the flowchart on page 190 of Salkind (2008) we see that the t test for dependent samples is the appropriate test statistic because we are comparing the means (of test scores) between two dependent groups (the same people tested twice, once before learning the method and once after. Open the dataset “Chapter6_example1.sav”. Take a moment to familiarize yourself with the data. Note how data for this type of analysis should be entered. 1) Each participant has one row in the data 2) One column is used to indicate that participant’s score on the pretest (i.e. the number of problems out of 10 they answered correctly) 3) Another column indicates each participant’s score on the posttest. The data should look something like this in SPSS:
If you were to switch to variable view, you would see that more descriptive labels have been added to the variables. These labels will be what shows up in the output and are helpful to remind you (the researcher) what the variables represent. In your own data set, you could use whatever labels you find most helpful.
4. Computation of the test statistic. We will use SPSS to compute the test statistic for us. To do so, click on the “Analyze” dropdown menu, highlight “Compare Means”, and then click on “Paired Samples t Test”, as pictured below.
The following pop-up window will appear:
Highlight the name of the variable that represents the pre-test score and click the arrow to place it in the “paired variables:” box. Next, highlight the name of the variable that designates the posttest score and click the arrow again to also place it in the “paired variables:” box. Your screen should look like the picture below:
Now, Click OK and navigate to the output window to find your results. The output will look like this (continued on the next 2 pages):
Paired Samples Statistics
Mean Pair Pretest Number of Correct Answers
Std.
Std.
Deviatio
Error
n
Mean
N
4.7000
10 1.76698
.55877
5.7000
10 1.63639
.51747
This table tells us the descriptive statistics (mean, N and standard deviation) for both the variables we tested (in our example, pre and posttest scores). These will be helpful in writing up our results.
1 Posttest Number of Correct Answers
Paired Samples Correlations N Pair 1
Pretest Number of Correct
Correlation 10
Sig.
.465
.176
Answers & Posttest Number of Correct Answers
This box tells us the value of the correlation coefficient and the p‐value associated with it for the two variables we tested. A correlation coefficient is a numerical index that tells us about the relationship between the two variables (if you haven’t learned about correlations yet, don’t worry ‐ you will learn about them later). For our purposes right now, we can ignore this box.
Paired Samples Test Paired Differences 95% Confidence Interval of the Difference
Std. Mean Pair 1
Pretest Number of Correct Answers -
-1.00000
Deviation
Std. Error Mean
1.76383
.55777
Lower -2.26177
Upper .26177
Sig. (2t -1.793
df
tailed) 9
.107
Posttest Number of Correct Answers
The box labeled “mean” tells us the difference between the two means (i.e. pretest mean – posttest mean OR 4.7 – 5.7). The box labeled “standard deviation” tells us the standard deviation of the difference scores (think the “Difference” column when you calculate this by hand). This is useful for calculating the effect size.
The box labeled “t” tells us the exact obtained value for our test statistic. This is the same number you would get if you calculated t by hand following the method in your text book. The box labeled “df” tells us the degrees of freedom for the test (i.e. n ‐1, where n equals the number of pairs). The box labeled “Sig. (2‐tailed) tells us the exact p‐value associated with this obtained value and number of degrees of freedom.
Interpreting the Output The third piece of the output (the table labeled “paired samples t-test”) includes all of the information you’ll need to complete the 8 steps. In particular, you want to locate the obtained value, degrees of freedom, and p-value associated with the analysis you just ran. There is also information that helps you interpret the t-statistic and some information that you can simply ignore. 5. Determination of the value needed for rejection of the null hypothesis If we were doing this example by hand, then this is the point when we would look at the table of critical values for t (Salkind, 2008, pgs 333-334). Recall that we look up the critical value to tell us the smallest value of t needed to reject the null hypothesis. The critical value is the t that corresponds to a p of .05 for a specific number of degrees of freedom. Because p gets smaller as t gets bigger, we know that if our obtained value is bigger than that critical value, then the p is less than .05. However, SPSS gives us an exact p-value! This means we don’t have to find the critical value on our own when we use SPSS. 6. Comparison of the obtained value and the critical value is made Because SPSS gives us a p-value, all we have to do now is see whether that p-value given to us (in the output) is greater or less than .05 (the level of risk we are willing to take). In this example, the output tells us that the p-value for our test is .107 (which is greater than our cutoff value of .05). This means that we fail to reject the null hypothesis. Put another way, this means there is a 10.7% chance that we would obtain this pattern of results if the null hypothesis was true and there was no effect of the method on algebra ability. While 10.7% might not seem very likely, recall that we are only willing to take a 5% chance. This means that we cannot reject that null hypothesis in our example. 7/8. Making a Decision Because our p-value (.107) is greater than .05, this means we fail to reject the null hypothesis. In other words, we conclude that there is no significant difference between the pretest and posttest scores. Interpretation of the Findings Now we report our results. Here’s an example of how these results would be reported in a journal article: A t test for dependent samples revealed that there was not a significant effect of the learning method on test scores (t(9) = -1.79, p > .05). Although participants performed somewhat better after (M = 5.70, SD = 1.63) compared to before the tutorial (M = 4.7, SD = 1.77), this difference was not statistically significant.
For someone unfamiliar with stats, you might say: The algebra tutorial does not appear to influence algebra ability. Effect Sizes Although not covered in Salkind (2008), it can be helpful to compute an effect size of the difference between the scores in your data set. For a refresher, see Salkind (2008) pages 178180 for a discussion of why effect sizes are important. To compute an effect size for dependent t, we can easily plug in some of the information from the output we already generated into the following formula: Mean Difference Standard Deviation of the Difference Scores If you refer to your output, you’ll see both of these pieces of information in the first part of the table labeled “paired samples t-test”. For our example then: 1 1.76
.57
**Notice that we’ve used a 1 rather than the -1 in the output. That’s because we are interested in the SIZE of the effect, not the direction. Cohen’s guidelines for interpreting an effect size (See page 180 of Salkind) tell us that this is actually a large effect size. This is somewhat contradictory with our non-significant results reported above. While the results of this particular study suggest that the pattern of results likely occurred by chance, this may also mean that all hope might not be lost for your proposed method! At this point, you might try running another study with either more participants or a different methodology. This helps illustrate why it can be important to pay attention to both statistical significance AND effect sizes. This doesn’t guarantee you’ll find a significant result the next time, but does suggest it might be worth trying. On the other hand, if the effect size had also been very small (i.e., close to zero) then you might want to think about developing a new method…
Practice Problem #1 (for SPSS) A researcher is interested in changes over time in happiness after a break up. He asks 6 people to complete a Happiness scale the first day after a break up and then complete the same scale again 20 days later. Higher numbers represent more happiness. Use SPSS to enter the data below and answer the questions that follow. Day 1
Day 20
12.00 14.00 13.00 15.00 13.00 15.00
15.00 18.00 15.00 18.00 13.00 17.00
A. What is the null and research hypothesis (in both words and statistical format)?
B. What is the level of risk associated with the null hypothesis? C. What is the appropriate test statistic and WHY?
D. What is the obtained value for the t-t test and what is its associated p-value? Is the difference between the two groups statistically significant?
E. What do you conclude about the effect of video game type on aggression? Write up your results as you would for a journal article.
F. Write up your results as you would for an intelligent person who doesn’t know stats.
Practice Problem #2 (Hand Calculation) The data below are from participants’ ratings of how jittery they feel before and after drinking a Red Bull (higher scores equal more jittery). Use a dependent samples t-test to assess whether or not the Red Bull increased feelings of jitteriness. Pretest 3 2 4 2 1 5 6 3 2 3
Posttest 4 7 6 4 5 8 9 8 6 7
A. What is the null and research hypothesis (in both words and statistical format)?
B. What is the level of risk associated with the null hypothesis? C. What is the appropriate t test statistic and WHY?
D. What is the value needed for rejection of the null hypothesis (i.e. the critical value)? E. What is the obtained value? Is it greater than the critical value?
F. Can you reject the null hypothesis? What do you conclude about the effectiveness of the pain relief drug? Write up the results as you would for a journal article. You should calculate the means of the two groups for this problem, but you do not have to calculate the standard deviation.
G. Write up your results as you would for an intelligent person who doesn’t know stats.
Practice Problem 3 (SPSS and Hand Calculation) Below are the number of free throws made (out of 5 attempts) by a group of players. During the first set of attempts, the crowd was trying to distract the players. During the second set of attempts, the crowd was quiet. Hand calculate the analysis using the data below and then use SPSS to check your answer. Also calculate an effect size for the data and include this information when you write up your results as you would for a journal article. Time1 Time2 2.00
1.00
3.00
4.00
3.00
3.00
4.00
5.00
4.00
3.00
5.00
3.00
