Exploring variation between studies: subgroup analysis
Subgroup analysis splits the data by a characteristic, such as age or sex, and compares the effect between groups, to ask why studies disagree.
Remember
When studies disagree, compare the effect between planned subgroups, such as age or sex, to see whether differences between the people studied explain why.
Studies of the same treatment can reach different results, and reviewers try to explain why. In a meta-analysis (a collation of similar studies collected to form one conclusion), results that vary by more than chance alone would produce are called heterogeneity. Heterogeneity is a clue, and reviewers use it to ask why the studies disagree.
Studies differ in who they recruit (the participants, meaning the people who take part), for example in the mix of age and sex, in the dose or timing of the treatment, and in how they were designed. If the true effect depends on a characteristic such as the average age of the participants, studies with different age profiles disagree for a real reason.
A subgroup analysis tests this directly. The data from the included studies are divided by a characteristic, such as being male or female or younger or older, and the effect in each group is compared. Another method is meta-regression, which relates the size of the effect to a study characteristic.
Treat a surprising subgroup finding as a lead to check rather than as proof. Many subgroup analyses make false positive findings (apparent findings that are not real) likely, so the analyses should be planned in advance and kept few.
A sensitivity analysis can check whether study methods change the result, but it does not test whether differences between participants explain the spread of effects. It repeats a review after changing a decision, such as which studies are included. A random-effects model (one that lets the true effects differ between studies) lets the summary reflect variation between studies, but it does not identify the source of that variation.
Key facts
- Subgroup analysis
- divides the data by a characteristic such as age or sex, then compares the effect in each group.
- Heterogeneity
- variation between study results that is larger than chance alone would produce.
- Effect modification
- means the effect differs from one group to another.
- Meta-regression
- a method that relates the size of the effect to a study characteristic.
- Caution
- many subgroup analyses make false positive findings likely, so plan them in advance and keep them few.
- Sensitivity analysis
- repeats a review after changing a decision, such as which studies are included.
Exam traps
- Standardising participants to one age and sex profile before the results are averaged tries to remove differences between participants but does not compare the effect across those groups, so it cannot show whether differences between participants explain the variation.
- A fixed-effect model does not explain differences between studies: it assumes every study estimates the same true effect, so it ignores heterogeneity instead of explaining it.
- Allowing for variation between studies with a random-effects model does not identify the source of that variation.
Read the full notes
Tested in 6 SimpleMed+ questions
Practice it in Population and Social Science, with the answer and a full explanation after every question.
Practice these questions