Skip to content
Scintria
ToolsGrant writerFunding
HomeLiteratureClinical trialsEvidence LabProtocol packData analysisLearnProjectsLibraryReferencesApplications
Start free
ScintriaStart free

Learn / Biostatistics / Sampling and estimation

Populations, samples and the ways studies go wrong

13 minute read · 6-question quiz

Download this lesson as a colour Word or PDF study guide with the figures and answers. Included in paid plans; reading and quizzes stay free.Sign in

What you will learn

  • Distinguish a population from a sample, and a parameter from a statistic

  • Compare simple random, systematic, stratified, cluster and convenience sampling

  • Separate random error (chance) from systematic error (bias) and name common sources of selection bias

A population is everyone you want to learn about: all pregnant women in Mbale district, all adults with type 2 diabetes in Kenya. A sample is the smaller group you actually study. A number that describes the population (the true mean blood pressure) is a parameter; the same number calculated from your sample is a statistic, our estimate of the parameter. Biostatistics is largely the discipline of saying how far a statistic is likely to be from its parameter.

Sampling methods

Simple randomStratifiedClusterConvenienceEveryone has anequal chanceNeeds a completelist (samplingframe)Draw with randomnumbersGold standard,often impracticalSplit into groups(strata) firstSample randomlywithin eachGuarantees smallgroups appearGives precisegroup estimatesSample wholeunits: villages,schoolsCheaper, no fulllist neededIndividuals in acluster are alikeNeeds a largersample (designeffect)Whoever is easy toreachQuick and cheapNo basis forgeneralisingHigh risk ofselection bias
Four ways to choose a sample. The first three are probability samples; the last is not.

Systematic sampling (every 5th patient through the door) is a convenient version of random sampling that works well unless the list has a hidden pattern. Multistage sampling combines methods: the Demographic and Health Surveys sample enumeration areas (clusters), then households within them.

Two kinds of error

  • Random error (sampling error) is chance. A different random sample would give a slightly different answer. It shrinks as the sample grows and is measured by the standard error and confidence intervals.
  • Systematic error (bias) is a flaw in design or measurement that pushes results in one direction. Making the sample bigger does not reduce it. A huge biased sample gives a very precise wrong answer.
Precise but biasedUnbiased but impreciseThe aimEstimates cluster tightly…around the wrong valueCause: flawed methodMore data does not helpRight on averageWide scatter betweensamplesCause: small sampleMore data fixes itTight and centredSound design plus adequatesizeReported with a confidenceintervalChecked against biassources
Precision (random error) and validity (bias) are separate problems with separate cures.

Common biases to recognise

  • Selection bias: the people studied differ systematically from those you want to describe. A survey conducted at a hospital outpatient clinic over-represents the sick and those who can afford to travel.
  • Non-response bias: people who decline or cannot be found differ from those who take part. If 40% of men are away from home at the time of a household visit, a survey of "heads of household" will miss them.
  • Loss to follow-up (attrition): in cohorts and trials, people who drop out often have worse outcomes. Analysing only completers can make a treatment look better than it is.
  • Survivorship and healthy-volunteer bias: people who join prevention studies are usually healthier than the general population.
  • Measurement and recall bias: instruments that are miscalibrated, or mothers who recall exposures differently after a child has become ill.

When a huge sample is still wrong

In 1936 a US magazine mailed ten million questionnaires, drawn from telephone directories and car registrations, and predicted the presidential election from the 2.4 million replies. A far smaller quota sample correctly predicted the opposite result. The large sample over-represented wealthy households (telephones and cars were luxuries) and the people who chose to reply. Size cannot cure a biased sampling method.

Ask these questions of any study

Who was eligible? Who was invited? Who agreed? Who completed? Compare the final group with the population you want to talk about. Every drop at each stage is a place where bias can enter, and good reports show it in a flow diagram.

Convenience is not randomness

'Random' in everyday speech means 'haphazard'. In statistics it means each unit has a known, non-zero probability of selection, set by a chance mechanism you control. Asking the next ten people who walk past is haphazard, not random.

Summary

  • Parameters describe populations; statistics describe samples and estimate parameters.

  • Probability sampling (simple random, stratified, cluster) allows valid generalisation; convenience sampling does not.

  • Random error shrinks with sample size; bias does not.

  • Selection, non-response and attrition biases are the most common threats; report how many were eligible, invited, enrolled and analysed.

Check your understanding

Question 1
1

A sample statistic is used to estimate:

Question 2
2

A survey of 20 villages chosen at random, with every household in them interviewed, is:

Question 3
3

Doubling the sample size will mainly reduce:

Question 4
4

A hospital-based study of malaria in children probably overestimates community severity because of:

Question 5
5

Why does stratified sampling help when a subgroup is small?

Question 6
6

Which is a probability sample?

Draft study guide. Spotted an error or something unclear? Tell us at hello@scintria.com. This is educational material, not clinical or regulatory advice.

The normal distribution and z-scoresSampling variation, standard error and the central limit theorem