Pre-specifying a primary outcome to guard against data-dredging
A primary outcome is the one main measure a trial is designed to answer, named in advance to guard against data-dredging (running many tests until one looks significant by chance).
Remember
Name one primary outcome in the written plan before the trial starts, so that a chance result from running many tests cannot be presented as the main finding.
Memory aid
draw the target before you fire the arrow, rather than painting a bullseye round wherever the arrow lands
The primary outcome is the target, fixed before the trial starts, and not a result picked out once the data are seen.
A randomised controlled trial (RCT, a trial in which patients are randomly allocated to either the new treatment or a comparator, meaning a placebo or the current standard) is the design of choice for testing whether a drug works. The trial's protocol (its written plan, ideally registered, meaning lodged publicly before the trial starts) names a single primary outcome: the one main measure the trial is designed to answer. Take a trial of an asthma drug: the primary outcome might be how far forced expiratory volume in one second (the volume of air a person can blow out in the first second of a hard breath) changes.
To pre-specify an outcome is to fix it in the protocol before any results exist. Further measures, such as symptom scores, are still allowed as secondary outcomes, but they are kept to a small number and named in advance too.
Pre-specifying outcomes guards against data-dredging, which is running many statistical tests until one comes back "significant" by chance, then presenting that result as if it had been the planned aim all along. A result counts as statistically significant when its p-value (the probability of seeing a result at least this extreme by chance if the drug truly had no effect) is below the conventional 0.05. Every extra test is another opportunity for a false positive, meaning an apparent effect that is really just chance.
Writing the primary outcome down first, ideally in a registered protocol, lets readers check that the main result reported is the one that was planned. An outcome picked after the results are known has not been pre-specified, whatever it is called.
Key facts
- Primary outcome
- the one main measure the trial is designed to answer, chosen before the trial starts.
- Secondary outcomes
- still allowed, but kept to a small number and named in advance.
- Data-dredging
- many tests run until one is "significant" by chance, giving a false positive.
- More tests, more chance
- each extra test raises the chance of at least one false positive.
- Size of the risk
- if a drug did nothing and 18 independent tests were each run at the 0.05 threshold, the chance that at least one would come out significant is roughly 60%.
- Registration
- publicly lodging the plan beforehand lets readers spot outcomes that were changed.
Exam traps
- Hiding which treatment people are receiving is blinding, which masks people so that expectations do not influence what is reported or recorded. Naming a primary outcome does nothing to hide who is receiving the drug.
- Spreading confounders (other factors that could distort the result) evenly between groups is the job of randomisation, which balances them on average. Choosing a primary outcome has no effect on how participants are allocated.
- Naming a primary outcome does not raise statistical power (the chance of detecting a real difference when one exists), because it does not recruit a single extra participant. Testing more outcomes would add chances of false positive findings instead.
Read the full notes
Tested in 6 SimpleMed+ questions
Practice it in Pharmacology, with the answer and a full explanation after every question.
Practice these questions