Skip to content
Scintria
ToolsGrant writerFunding
HomeLiteratureClinical trialsEvidence LabProtocol packData analysisLearnProjectsLibraryReferencesApplications
Start free
ScintriaStart free

Learn / Biostatistics / Statistics in clinical trials and evidence synthesis

Interim analyses, data monitoring and multiplicity in trials

14 minute read · 6-question quiz

Download this lesson as a colour Word or PDF study guide with the figures and answers. Included in paid plans; reading and quizzes stay free.Sign in

What you will learn

  • Explain why repeated looks at accumulating data inflate the false-positive rate

  • Describe group sequential boundaries (Pocock, O'Brien-Fleming) and the role of a DSMB

  • Handle multiple primary outcomes, arms and subgroups in a hierarchy

Large trials accumulate data for years. It is tempting, and ethically necessary, to look at the results before the end: if a treatment is harming people, or clearly working, continuing wastes lives. But each look at the data is another chance for a false positive.

The problem with peeking

If you test at p < 0.05 after every quarter of enrolment (4 looks) and stop the first time it is reached, the overall false-positive rate rises from 5% to about 13%. With 10 looks it is about 19%. The curves of the test statistic wander, and with enough looks even a null treatment will cross the line at some point.

05101520Overall false-positive rate (%)5%1 look8%2 looks11%3 looks13%4 looks14%5 looks19%10 looks
Testing at 0.05 at each of k equally spaced looks and stopping at the first success inflates the type I error.

Group sequential designs

The solution is to plan the looks in advance and make the criterion for stopping stricter at each interim look, so that the overall type I error remains 5%.

BoundaryIdeaNominal two-sided cut-offs for 4 equally spaced looks (final at look 4)
PocockThe same stringent cut-off at every lookz = 2.36 at each look (p = 0.018)
O'Brien-FlemingVery strict early, close to conventional at the endz ≈ 4.05, 2.86, 2.34, 2.02 (p ≈ 0.00005, 0.004, 0.019, 0.043)
Lan-DeMets alpha-spendingA flexible version: a function says how much α is 'spent' by each fraction of information, so looks can be unplanned in timingMimics Pocock or O'Brien-Fleming

O'Brien-Fleming boundaries are the most common in practice. They make early stopping for efficacy rare unless the effect is overwhelming, and they leave nearly the full α for the final analysis, so the final sample size is only slightly inflated (about 2% to 3%).

Stopping for futility and harm

Interim analyses may also stop a trial because it is unlikely to show benefit (futility, using conditional power) or because the treatment is harmful (safety boundaries, often more liberal than efficacy ones). Stopping early for benefit tends to overestimate the effect, so trials stopped early need cautious interpretation.

The Data and Safety Monitoring Board (DSMB / DMC)

An independent group of clinicians, a statistician and often an ethicist who review unblinded accumulating data and advise the sponsor and steering committee whether to continue, modify or stop.

  • Has a written charter defining membership, meeting schedule, stopping rules and confidentiality.
  • Sees data by arm; the trial team stays blinded.
  • Recommends only; the sponsor or steering committee decides.
  • Required for most large, multi-site trials, especially with mortality or serious morbidity outcomes.
  • Independent statisticians produce the unblinded reports.

Multiplicity: more than one primary question

Besides repeated looks, trials create multiplicity through several outcomes, several treatment arms, several time points and subgroups. Strategies:

  1. Single primary outcome. The best protection; everything else is secondary or exploratory.
  2. Hierarchical (fixed-sequence) testing: test outcomes in a pre-specified order, each at α = 0.05, and stop at the first non-significant one. No α loss, but later outcomes depend on earlier success.
  3. Bonferroni or Holm across co-primary outcomes or arms (multiple doses versus control).
  4. Gatekeeping procedures for families of endpoints.
  5. Subgroups: limit and pre-specify; test for interaction, not within-subgroup p-values.

Hierarchy in practice

A malaria vaccine trial pre-specifies: (1) first clinical malaria episode at 12 months (α = 0.05); only if significant, (2) severe malaria; only if significant, (3) all-cause hospitalisation. If (1) fails, no claims are made about (2) or (3), however small their p-values. This protects the overall false-positive rate at 5% and removes any temptation to pick the favourable result afterward.

Adaptive designs need extra care

Adaptive trials (sample-size re-estimation, dropping arms, response-adaptive randomisation, seamless phase 2/3) save time and participants but must be pre-specified with simulations that show control of type I error. Unplanned changes after seeing unblinded data compromise the whole trial. Regulators (FDA, EMA) publish guidance on adaptive designs.

What a protocol should say

Number and timing of interim analyses; the stopping boundaries and spending function; who sees unblinded data; the DSMB charter; and the hierarchy for multiple outcomes.

Summary

  • Each interim look at the data inflates the false-positive rate unless boundaries are adjusted.

  • O'Brien-Fleming is strict early and nearly conventional at the end; Pocock is constant.

  • An independent DSMB reviews unblinded data and recommends continue, modify or stop.

  • Control multiplicity with one primary outcome, fixed-sequence testing, Bonferroni/Holm and pre-specified subgroups.

  • Adaptive designs must be pre-specified and simulated.

Check your understanding

Question 1
1

Testing at p < 0.05 after each of 4 looks and stopping at the first success gives an overall false-positive rate of about:

Question 2
2

O'Brien-Fleming boundaries are characterised by:

Question 3
3

Who should see unblinded interim data by arm?

Question 4
4

Under a fixed-sequence hierarchy, if the first outcome is not significant, then:

Question 5
5

A trial stopped early for benefit tends to:

Question 6
6

Which is the best defence against multiple outcomes inflating false positives?

Draft study guide. Spotted an error or something unclear? Tell us at hello@scintria.com. This is educational material, not clinical or regulatory advice.

Intention-to-treat, per-protocol and missing dataMeta-analysis, heterogeneity and reporting standards