Learn / Biostatistics / Statistics in clinical trials and evidence synthesis
Interim analyses, data monitoring and multiplicity in trials
14 minute read · 6-question quiz
What you will learn
Explain why repeated looks at accumulating data inflate the false-positive rate
Describe group sequential boundaries (Pocock, O'Brien-Fleming) and the role of a DSMB
Handle multiple primary outcomes, arms and subgroups in a hierarchy
Large trials accumulate data for years. It is tempting, and ethically necessary, to look at the results before the end: if a treatment is harming people, or clearly working, continuing wastes lives. But each look at the data is another chance for a false positive.
The problem with peeking
If you test at p < 0.05 after every quarter of enrolment (4 looks) and stop the first time it is reached, the overall false-positive rate rises from 5% to about 13%. With 10 looks it is about 19%. The curves of the test statistic wander, and with enough looks even a null treatment will cross the line at some point.
Group sequential designs
The solution is to plan the looks in advance and make the criterion for stopping stricter at each interim look, so that the overall type I error remains 5%.
| Boundary | Idea | Nominal two-sided cut-offs for 4 equally spaced looks (final at look 4) |
| Pocock | The same stringent cut-off at every look | z = 2.36 at each look (p = 0.018) |
| O'Brien-Fleming | Very strict early, close to conventional at the end | z ≈ 4.05, 2.86, 2.34, 2.02 (p ≈ 0.00005, 0.004, 0.019, 0.043) |
| Lan-DeMets alpha-spending | A flexible version: a function says how much α is 'spent' by each fraction of information, so looks can be unplanned in timing | Mimics Pocock or O'Brien-Fleming |
O'Brien-Fleming boundaries are the most common in practice. They make early stopping for efficacy rare unless the effect is overwhelming, and they leave nearly the full α for the final analysis, so the final sample size is only slightly inflated (about 2% to 3%).
The Data and Safety Monitoring Board (DSMB / DMC)
An independent group of clinicians, a statistician and often an ethicist who review unblinded accumulating data and advise the sponsor and steering committee whether to continue, modify or stop.
- Has a written charter defining membership, meeting schedule, stopping rules and confidentiality.
- Sees data by arm; the trial team stays blinded.
- Recommends only; the sponsor or steering committee decides.
- Required for most large, multi-site trials, especially with mortality or serious morbidity outcomes.
- Independent statisticians produce the unblinded reports.
Multiplicity: more than one primary question
Besides repeated looks, trials create multiplicity through several outcomes, several treatment arms, several time points and subgroups. Strategies:
- Single primary outcome. The best protection; everything else is secondary or exploratory.
- Hierarchical (fixed-sequence) testing: test outcomes in a pre-specified order, each at α = 0.05, and stop at the first non-significant one. No α loss, but later outcomes depend on earlier success.
- Bonferroni or Holm across co-primary outcomes or arms (multiple doses versus control).
- Gatekeeping procedures for families of endpoints.
- Subgroups: limit and pre-specify; test for interaction, not within-subgroup p-values.
Summary
Each interim look at the data inflates the false-positive rate unless boundaries are adjusted.
O'Brien-Fleming is strict early and nearly conventional at the end; Pocock is constant.
An independent DSMB reviews unblinded data and recommends continue, modify or stop.
Control multiplicity with one primary outcome, fixed-sequence testing, Bonferroni/Holm and pre-specified subgroups.
Adaptive designs must be pre-specified and simulated.
Check your understanding
Draft study guide. Spotted an error or something unclear? Tell us at hello@scintria.com. This is educational material, not clinical or regulatory advice.