Skip to content
Scintria
ToolsGrant writerFunding
HomeLiteratureClinical trialsEvidence LabProtocol packData analysisLearnProjectsLibraryReferencesApplications
Start free
ScintriaStart free

Learn / Biostatistics / Statistics in clinical trials and evidence synthesis

Meta-analysis, heterogeneity and reporting standards

15 minute read · 6-question quiz

Download this lesson as a colour Word or PDF study guide with the figures and answers. Included in paid plans; reading and quizzes stay free.Sign in

What you will learn

  • Pool results of several trials with fixed- and random-effects methods and read a forest plot

  • Interpret heterogeneity (Q, I²) and publication bias

  • Name the main reporting guidelines (CONSORT, STROBE, PRISMA, TRIPOD, SPIRIT) and what they require

A single trial rarely settles a question. A systematic review finds all relevant studies by a reproducible method, appraises them and summarises them. A meta-analysis statistically combines their results into one estimate with greater precision.

Pooling: the weighted average

Each study gives an effect estimate (often a log risk ratio, log odds ratio or mean difference) and its standard error. The pooled estimate is a weighted average where more precise studies get more weight.

  • Fixed-effect (inverse-variance) model: assumes all studies estimate one common true effect; weight = 1/SE².
  • Random-effects model (DerSimonian-Laird or REML): assumes each study's true effect varies around an average, with between-study variance τ²; weight = 1/(SE² + τ²). It gives smaller studies relatively more weight and produces wider intervals. Preferred when studies differ in populations or settings.

Worked example: five trials of an intervention on malaria

TrialRR95% CIWeight (random)
Kampala 20150.700.45 to 1.0916.3%
Kisumu 20170.550.35 to 0.8615.9%
Dar es Salaam 20180.850.60 to 1.2023.5%
Blantyre 20190.620.40 to 0.9616.6%
Lusaka 20210.950.70 to 1.2927.8%

Fixed-effect pooled RR = 0.77 (95% CI 0.65 to 0.91). Random-effects pooled RR = 0.75 (95% CI 0.62 to 0.92). Individually, three trials are borderline or non-significant at the 5% level; together the evidence is clear. Heterogeneity: Q = 5.39 on 4 df (p = 0.25), I² = 26%.

StudyEstimate (95% CI)Kampala 20150.7 (0.45–1.09)Kisumu 20170.55 (0.35–0.86)Dar es Salaam 20180.85 (0.6–1.2)Blantyre 20190.62 (0.4–0.96)Lusaka 20210.95 (0.7–1.29)Pooled (random effects)0.75 (0.62–0.92)0.51Risk ratio (log scale); below 1 favours the intervention
A forest plot: each square is a trial (size shows weight), each line its CI; the diamond is the pooled random-effects estimate.

Heterogeneity

Are the trials telling the same story? Look at the forest plot: do the intervals overlap? Then quantify.

  • Cochran's Q: tests whether the variation exceeds chance. Low power with few studies.
  • I²: the percentage of total variability due to genuine differences between studies rather than chance. Rough guide: 25% low, 50% moderate, 75% high (Higgins).
  • τ²: the between-study variance on the effect scale.
  • Prediction interval: the range in which the true effect of a new study is expected to fall. Wider than the CI of the pooled mean, and more honest when heterogeneity exists.

Explore causes with subgroup analysis and meta-regression (dose, setting, risk of bias), recognising that they are observational comparisons between trials. If studies are too different, don't pool.

Bias in meta-analysis

  • Publication bias: positive trials are published more often. A funnel plot (effect against precision) should be symmetric; asymmetry suggests small-study effects. Test with Egger's test only when there are at least ~10 studies.
  • Risk of bias in included trials: use the Cochrane RoB 2 tool (randomisation process, deviations, missing data, outcome measurement, selective reporting). Pooling biased trials gives a precise, biased answer.
  • GRADE rates the certainty of the body of evidence (high, moderate, low, very low).

Garbage in, garbage out

Meta-analysis is not magic. It improves precision, not validity: combining poor-quality studies produces a narrow CI around a wrong answer. Always report the number of studies, the risk of bias and the heterogeneity next to the pooled estimate.

Reporting guidelines: say what you did, so others can judge

GuidelineForKey items
CONSORT 2010Randomised trialsFlow diagram of participants; primary outcome and sample size; randomisation and blinding; ITT; effect size with CI; harms; registration
SPIRITTrial protocolsObjectives, design, eligibility, interventions, outcomes, sample size, analysis plan, DSMB
STROBEObservational studiesSetting, selection, confounding, bias, missing data
PRISMA 2020Systematic reviewsSearch strategy, selection flow, risk of bias, synthesis methods
TRIPODPrediction modelsDevelopment, validation, performance
STARDDiagnostic accuracySensitivity and specificity with CIs; flow of participants
CHEERSEconomic evaluationsCosts, outcomes, perspective

Most journals require the relevant checklist. Use the EQUATOR Network (equator-network.org) to find any guideline. Follow it from the start, when writing the protocol, and not just when preparing a manuscript.

Register and share

Register trials before enrolment (ClinicalTrials.gov, ISRCTN, PACTR for Africa, WHO ICTRP), publish a statistical analysis plan, and share de-identified data where possible. Registration makes selective outcome reporting detectable.

The tests don't replace judgment

A meta-analysis with I² = 0% can still be misleading if all trials share the same bias. And an I² of 80% does not always forbid pooling: sometimes the direction is consistent and the size varies. Report the heterogeneity, explore it, and describe how confident you are.

Summary

  • Meta-analysis pools study estimates with inverse-variance weights; random-effects allows true effects to vary.

  • Read the forest plot: squares for studies, diamond for the pooled estimate, line of no effect.

  • Heterogeneity: Q, I² (percentage of variation beyond chance) and τ²; use prediction intervals.

  • Publication bias, risk of bias and certainty (GRADE) shape how far to trust the result.

  • Use CONSORT, SPIRIT, STROBE, PRISMA, TRIPOD, STARD and register trials in advance.

Check your understanding

Question 1
1

In a random-effects meta-analysis, compared with fixed-effect, small studies generally receive:

Question 2
2

I² = 75% indicates:

Question 3
3

The diamond in a forest plot shows:

Question 4
4

A funnel plot that is clearly asymmetric suggests:

Question 5
5

Which guideline governs the reporting of a randomised trial?

Question 6
6

Pooling ten poorly conducted, biased trials is likely to give:

Draft study guide. Spotted an error or something unclear? Tell us at hello@scintria.com. This is educational material, not clinical or regulatory advice.

Interim analyses, data monitoring and multiplicity in trials