Learn / Epidemiology / Evaluating interventions
Natural experiments and quasi-experimental evaluations
13 minute read · 6-question quiz
What you will learn
Explain when randomisation is not possible and what alternatives exist
Calculate a difference-in-differences estimate and state its key assumption
Describe an interrupted time series and a regression discontinuity design
Many important public health decisions cannot be randomised: national laws, a sugar tax, a district-wide programme, a pandemic lockdown. Quasi-experimental methods use non-random comparisons designed to approximate the counterfactual.
Difference-in-differences (DiD)
Compare the change over time in the intervention group with the change in a comparison group. The comparison group's change estimates what would have happened to the intervention group without the intervention.
Interrupted time series (ITS)
When there is no comparison group, use a long series of measurements before and after the intervention. The analysis (segmented regression) estimates the level change at the intervention and the change in slope, after allowing for the pre-existing trend. Typically 8 to 12 or more observations before and after are recommended, with allowance for seasonality and autocorrelation. A controlled ITS adds a comparison series.
Use: effect of a smoking ban on hospital admissions; a stock-out of antimalarials on monthly cases.
Regression discontinuity (RD)
Where eligibility is set by a cut-off on a continuous score (CD4 below 350 for ART in earlier guidelines, an income threshold for cash transfers), compare people just above and just below the threshold. They are almost identical except for treatment, so it approximates randomisation locally.
Other methods
- Propensity score matching or weighting: balance measured confounders between treated and untreated.
- Instrumental variables: use something that affects treatment but not the outcome directly (distance to clinic).
- Synthetic control: build a weighted combination of untreated areas that matches the treated area before the intervention.
- Natural experiments: exposure assigned by circumstances as-if randomly (a lottery for places, a change in policy at a border).
Summary
Quasi-experimental methods approximate the counterfactual when randomisation is not possible.
DiD compares changes over time between groups and assumes parallel trends.
ITS uses many time points to estimate level and slope changes.
Regression discontinuity compares people just above and below an eligibility cut-off.
State and test assumptions; use TREND and STROBE reporting.
Check your understanding
Draft study guide. Spotted an error or something unclear? Tell us at hello@scintria.com. This is educational material, not clinical or regulatory advice.