Student's t-Test of Difference of Means

Key Concepts and Terms

• Student's t-test The t-test is a parametric test assuming a normal distribution, but when its assumptions are met it is more powerful than corresponding two-sample nonparametric tests. The t-test is used when sample sizes are small (ex., < 30), but with larger samples the normal curve z test is used. The two tests are equivalent. Computation of t differs for independent vs. dependent samples, but inference is the same. With polytomous independents, researchers use ANOVA.

  Formula. Let E be the experimental condition and let C be the control condition. Let m be the means, s the standard deviations, and n be the sample size. Then
  

  t = (m_E - m_C) / SQRT[(s²_E + s²_C) / n ]
  

  for the case of equal sample sizes, n, in both groups. See Levin (1999: 25) for the formula when sample sizes differ.

• The critical value is the value found in a "Table of Values of the t Distribution", for a given level of significance (ex., .05). If the observed t is above the critical value t, then the difference of means is significant at that level.

• Confidence limits set upper and lower bounds on an estimate for a given level of significance (ex., the .05 level). The confidence interval is the range within these bounds. For instance, for normally distributed data, the confidence limits for an estimated mean are the sample mean plus or minus 1.96 times the standard error, as discussed in the section on normal curve z-tests. Some researchers recommend reporting confidence limits wherever point (ex., mean) estimates and their significance are reported. This is because confidence limits provide additional information on the relative meaningfulness of the estimates. Thus significance has a different meaning when, for example, the confidence interval is the entire range of the data, as compared to the situation where the confidence interval is only ten percent of the range. By default, SPSS prints out the 95% confidence interval for t-test differences between the sample mean and the hypothesized mean. However, by clicking the "Options" button in the "One-Sample T-Test" dialog box in SPSS, the researcher can customized the confidence level to something other than 95%.

• SPSS menu selections. When one is comparing a mean from a single sample with a known mean, select Analyze, Compare Means, One-Sample T-Test. If two samples are involved, select either Independent-Samples T-Test or Paired Samples T-Test, dependent on whether the samples are independent or correlated. One-Way ANOVA is also available from the same sub-menu.
  

• Types of t-tests:
  1. One-sample t-tests test whether the mean of one variable differs from a constant (ex., does the mean grade of 72 for a sample of students differ significantly from the passing grade of 70?). When p<.05 the researcher concludes the group mean is significantly different from the constant. In SPSS, select Analyze, Compare Means, One-Sample T-Test.
     
     • Interpretation. In the "One-Sample Test" table shown above, a sample's mean on years of education is compared with the hypothesis that the mean is 12. The finding of significance indicates that the sample mean is significantly different from 12 years. The "Mean Difference" of 1.251 means the sample mean was 12 + 1.251 = 13.251. That is, the sample mean of 13.251 is found to be significantly different from the hypothesized mean of 12 being tested.

  2. Independent sample t-tests are used to compare the means of two independently sampled groups (ex., do those working in high noise differ on a performance variable compared to those working in low noise, where individuals are randomly assigned to the high-noise or low-noise groups?) . When p<.05 the researcher concludes the two groups are significantly different in their means. From SPSS, select Analyze, Compare Means, Independent Samples T-Test; select the grouping variable (ex., 1 = high noise group, 2 = low noise group); select the test variable(s); set the confidence limits using the Options button (95% is default); OK.
     
     • Interpretation. As illustrated above, in SPSS the "Independent Samples Test" table contains the significance findings for differences in means between groups. In the example, this is the difference between means in income between males and females. Two tests are provided - one assuming equal variances between genders and the other assuming unequal variances. Assuming equal variances is the most common research choice. For these data, there is a finding of significance at the .000 level by either variance assumption. Significance means the researcher rejects the null hypothesis that the group means are equal, concluding instead that they are significantly different.

     • Warning. This test is often used to compare the means of two groups in the same sample (ex., men vs. women) even though individuals are not (in the case of gender, cannot be) assigned randomly to the two groups (to "men" and to "women"). Random assignment would have controlled for unmeasured variables. This opens up the possibility that other variables either mask or enhance any apparent significant difference in means. That is, the independent sample t-test tests the uncontrolled difference in means between two groups If a significant difference is found, it may be due not just to gender; control variables may be at work. The researcher will wish to introduce control variables, as in any multivariate analysis.

  3. Paired sample t-tests compare means where the two groups are correlated, as in before-after, repeated measures, matched-pairs, or case-control studies (ex., mean candidate evaluations before and after hearing a speech by the candidate). The algorithm applied to the data is different from the independent sample t-test, but interpretation of output is otherwise the same. In SPSS, select Analyze, Compare Means, Paired Sample T-Test; select two paired variables (use shift-select; the "before" and "after" responses must be organized as two variables in the data set; for a matched-pairs or case-control study, the response for each test subject and its matched control subject must be in the same case in the data file). From SPSS, select Analyze, Compare Means, Paired Samples T-Test
     
     • Interpretation. In the example above, the mean of Test is 2.318 points greater than the mean of Pretest, and this difference is significant at the .000 level, taking into account the fact that Test and Pretest are related rather than independent variables.

     • Example. A use of paired t-tests to compare which regression model has the lowest mean absolute residuals and thus is the best model, is illustrated in a section of the module on curve fitting.

• Effect size measures. When reporting significance levels, it is desirable to report effect size measures as well. Cohen's d is among the most common effect size measures for t-tests. It is covered in the section on ANOVA.

• One-way ANOVA is an alternative to test difference of means between independent samples. It is discussed separately and is found in SPSS under Analyze, Compare Means, One-Way ANOVA.

• Repeated measures ANOVA is also discussed separately and is an alternative to testing the difference of means for paired samples. It has been replaced in SPSS by GLM (General Linear Model) repeated measures. This procedure will enable the researcher to introduce other categorical factors and analyze main effects and interaction effects. The researcher will also be able to introduce continuous variables as covariates and see how these relate to the difference of means observed in the dependent variable. In SPSS, select Analyze, General Linear Model, Repeated Measures.

• The binomial test is an exact test which should be used in lieu of the t-test when sample size is small. It is supported by SPSS in the NPAR TESTS procedure.

Assumptions

• Approximately normal distribution of the measure in the two groups is assumed. There are tests for normality. The t-test may be unreliable when the two samples come from widely different shaped distributions (see Gardner, 1975). Moore (1995) suggests data for t-tests should be normally distributed for sample size less than 15, and should be approximately notmal and without outliers for samples between 15 and 40; but may markedly skewed when sample size is greater than 40.

• Roughly similar variances There is a test for homogeneity of variance, also called a test of homoscedasticity. In SPSS homogeneity of variances is tested by "Levene's Test for Equality of Variances", with F value and corresponding significance. These are part of SPSS output for two independent sample t-tests. There are also other tests for homogeneity of variances. The t-test may be unreliable when the two samples are unequal in size and also have unequal variances (see Gardner, 1975).

• Dependent/independent samples. The samples may be independent or dependent (ex., before-after, matched pairs). However, the calculation of t differs accordingly. SPSS offers the choice as "Independent Samples T Test" versus "Paired Samples T Test." In the one-sample test, it is assumed that the observations are independent.

Frequently Asked Questions

• How do you compute the t-test for a difference of correlations?
  This is discussed in the section on correlation.

• Where is the t-test of difference of means in the SPSS output?
  If not clearly labeled, in some packages, such as older versions of SPSS, the t value lies under the label "Pooled Variance Estimate".
  
  
• My statistics package gives me a choice between Levene's test and Bartlett's test to test homogeneity of variances. Which should I use?
  Levene's test is the default for most statistics packages because it is fairly robust when its normality assumption is violated, whereas Bartlett's test and F-tests of differences in variances are not.
  
  
• What non-parametric test do I use instead of the t-test if my data cannot meet the assumption of normality?
  The Mann-Whitney U test, also known as the Wilcoxon test, is seen by some researchers as a nonparametric analog to the t-test for variables with an even number of categories (leaving out the neutral category), and the Kruskal Wallis test if the neutral category is not excluded. There is no unambiguous answer to this question, however, and the researcher must analyzed the assumptions of particular nonparametric tests in relation to the research data at hand.
  
  

Bibliography

• Thomas A. Denes and Dames & Moore, Inc. (1997). Do small business set-asides increase the cost of government contracting? Public Administration Review, Vol. 57, No. 5 (Sept./Oct.): 441-444. Provides a short, useful illustration of the use of t-tests in public administration literature. Mean bids for set-aside and for unrestricted contracts in procurement of dredging services showed no significant difference by t-testing.
• Gardner, P. L. (1975). Scales and statistics. Review of Educational Research. 45: 43-57. Discusses assumptions of the t-test.
• Levin, Irwin P. (1999). Relating statistics and experimental design. Thousand Oaks, CA: Sage Publications. Quantitative Applications in the Social Sciences series #125. Elementary introduction covers t-tests and various simple ANOVA designs. Some additional discussion of chi-square, significance tests for correlation and regression. and non-parametric tests such as the runs test, median test, and Mann-Whitney U test.
• Moore, D. S. (1995). The basic practice of statistics. NY: Freeman and Co.