Interpreting a confidence interval against the null value
A result is not statistically significant when its confidence interval, the range of values compatible with the data, includes the null value, the number meaning no difference.
Remember
Judge significance from where the confidence interval sits against the null value, not from the point estimate (the best single estimate from the study). If the interval includes the null value, the result is not statistically significant, which does not prove there is no effect.
A confidence interval is the range of values that are compatible with the data. Compare that range with the null value, the number that means no difference.
The null value follows the scale. A ratio compares one group with another by division, as the relative risk does (the risk in one group compared with the risk in the other) and as the odds ratio does (an estimate of the strength of the association between an exposure and a disease). Equal numbers give a ratio of 1, so 1 is the null value for a ratio. For a difference, where one number is subtracted from the other, the null value is 0. When a study estimates a proportion, its null value is the proportion expected if nothing special were going on.
If the interval includes the null value, the data remain compatible with no difference. The null hypothesis (the idea that there is no effect) cannot be rejected, so the result is not statistically significant. If the whole interval lies on one side of the null value, the result is statistically significant.
The point estimate, the best single estimate from the study, does not decide this. A relative risk of 1.4 with an interval of 0.9 to 2.1 sits above 1, yet the interval includes 1, so the result is not statistically significant. Width does not decide it either: it shows precision (how exact the estimate is). A result that is not statistically significant does not prove there is no effect, because the interval usually also includes values showing benefit and values showing harm.
Key facts
- Null value for a ratio
- 1, because equal numbers give a ratio of 1.
- Null value for a difference
- 0, because subtracting equal numbers gives no difference.
- Interval includes the null value
- the result is not statistically significant.
- Interval excludes the null value
- the result is statistically significant.
- Point estimate
- does not decide significance; the interval does.
- Width of the interval
- shows precision, not significance.
- Not significant
- does not prove that there is no effect.
Exam traps
- Zero is the wrong test for an odds ratio, because zero is the null value for differences. An odds ratio cannot be negative, so its interval never includes zero, and the test is whether the interval includes 1.
- A point estimate above 1 does not make a result significant, because significance depends on the interval. If the interval includes 1, the true odds ratio could be 1 or even below it.
- A result that is not significant does not show that the exposure and the outcome are unrelated, because the interval can still include values that would mean a real difference, particularly when it is wide.
Read the full notes
Tested in 27 SimpleMed+ questions
Practice it in Population and Social Science, with the answer and a full explanation after every question.
Practice these questions