Skip to content
Scintria
ToolsGrant writerFunding
HomeLiteratureClinical trialsEvidence LabProtocol packData analysisLearnProjectsLibraryReferencesApplications
Start free
ScintriaStart free

Learn / Biostatistics / Hypothesis testing

Comparing means: the t-tests

15 minute read · 6-question quiz

Download this lesson as a colour Word or PDF study guide with the figures and answers. Included in paid plans; reading and quizzes stay free.Sign in

What you will learn

  • Choose between a one-sample, two-sample (independent) and paired t-test

  • Carry out and interpret a two-sample and a paired t-test from summary data

  • Check the assumptions and know what to do if they fail

The t-test compares means. It uses the sample SD to estimate the standard error and refers the test statistic to a t distribution (fatter tails than the normal, especially for small samples).

Which t-test?

1One group vs aknown valueOne-sample t-test2Two separategroupsIndependent(two-sample) t-test3Same peoplemeasured twice, ormatched pairsPaired t-test4Three or moregroupsAnalysis of variance(ANOVA)
Choose by the design, not by the numbers.

Test statistics

Independent groups: t = (x_1 - x_2)/(SE), SE = s_psqrt((1)/(n_1) + (1)/(n_2)), s_p = sqrt(((n_1 - 1)s_1^2 + (n_2 - 1)s_2^2)/(n_1 + n_2 - 2)), with n_1 + n_2 - 2 degrees of freedom.

Paired: compute the difference for each pair, then t = (d)/(s_d/sqrt(n)) with n - 1 degrees of freedom.

Worked example 1: two independent groups

Eighty adults with hypertension are randomised to a new regimen (A) or usual care (B). After 12 weeks:

GroupnMean SBPSD
A (new)40128.414.2
B (usual)40135.115.0
  1. Difference: 128.4 - 135.1 = -6.7 mmHg.
  2. Pooled SD: s_p = 14.6. SE = 14.6 × sqrt(1/40 + 1/40) = 3.27.
  3. t = -6.7 / 3.27 = -2.05, with 78 df.
  4. p = 0.044.
  5. 95% CI for the difference: -6.7 ± 1.99 × 3.27, i.e. -13.2 to -0.2 mmHg.

Report: "SBP was 6.7 mmHg lower with the new regimen (95% CI -13.2 to -0.2; p = 0.044)." Note how close the upper limit is to zero: the result is only just significant, and the plausible effect ranges from almost nothing to a large benefit.

Worked example 2: paired data

Twelve patients have SBP measured before and after a 4-week salt-reduction programme. The mean change is -6.5 mmHg and the SD of the changes is 8.2.

  • SE = 8.2 / sqrt(12) = 2.37
  • t = -6.5 / 2.37 = -2.75, 11 df
  • p = 0.019; 95% CI -11.7 to -1.3 mmHg

Using the paired test is more powerful than comparing two groups, because each person acts as their own control and between-person variability cancels. Treating paired data as two independent samples throws away this advantage and gives the wrong answer.

Assumptions

  1. Independent observations (apart from deliberate pairing). Violated by repeated measures on the same person or by clusters. This is the most serious assumption.
  2. Approximately normal data in each group, or normally distributed differences for the paired test. Matters mostly for small samples (under about 30 per group); the central limit theorem protects larger ones.
  3. Similar variances in the two groups for the pooled version. If SDs differ markedly, use Welch's t-test, which does not pool and adjusts the degrees of freedom. Many statisticians now use Welch by default.

What to do if assumptions fail

  • Skewed positive data: log-transform and analyse the logs (results are ratios of geometric means).
  • Small samples with skew or outliers: use the Wilcoxon / Mann-Whitney tests (next lesson).
  • Clustered or repeated data: use methods designed for them, such as mixed models or GEE.

Do not screen with a t-test, then reuse the data

Decide the analysis in the protocol. Choosing the test after seeing which gives p < 0.05 inflates the false-positive rate.

Three or more groups

Do not run several t-tests between pairs of groups. With 4 groups there are 6 comparisons, and the chance of at least one false positive is about 1 - 0.95^6 = 26\%. Use ANOVA for an overall test, followed by corrected pairwise comparisons.

100120140160SBP (mmHg)A: new regimenB: usual care
A box plot of each group is a quick way to see the difference, spread and any outliers before testing. (Illustrative data consistent with the table above.)

Summary

  • One-sample, independent and paired t-tests match different designs; pick by design.

  • t = difference ÷ SE; compare with a t distribution with the correct degrees of freedom.

  • Always report the difference with its 95% CI and the p-value.

  • Paired analysis uses within-person differences and is more powerful when pairing is real.

  • Use Welch for unequal SDs; transform or use rank tests for strong skew in small samples.

Check your understanding

Question 1
1

The same 30 patients are weighed before and after a diet. The right test is:

Question 2
2

A two-sample t-test gives t = 2.30 with 60 df. Roughly what is p (two-sided)?

Question 3
3

Group SDs are 10 and 30 with equal n. A better choice than the pooled t-test is:

Question 4
4

Four treatment groups are to be compared on mean blood pressure. The best first step is:

Question 5
5

In the worked example, the CI for the difference was -13.2 to -0.2. This means:

Question 6
6

Which violation is most serious for a t-test?

Draft study guide. Spotted an error or something unclear? Tell us at hello@scintria.com. This is educational material, not clinical or regulatory advice.

The logic of a significance testRank tests, multiple comparisons and the misuse of p-values