A confidence interval gives the range of values compatible with the data collected. A wide interval says the study could not pin the answer down; a narrow one says it could.
A confidence interval gives the range of values compatible with the data collected. A wide interval says the study could not pin the answer down; a narrow one says it could. Neither says the effect matters to anyone, and that is a separate judgement the statistics cannot make for a reader.
Headlines quote the middle of the range because it is a single figure that fits in a sentence. The middle is simply the most likely value within a span of plausible ones, and reporting it alone discards the uncertainty the study was run to measure.
If an interval around a difference includes zero, the data are compatible with there being no effect at all. That is not proof none exists — it means this study could not distinguish one, which is a statement about the study as much as about the world.
A 95% interval does not mean there is a 95% chance the true value lies inside it. It means a procedure repeated many times would capture the true value 95% of the time. The distinction sounds academic and changes how much weight a single published interval deserves.
More observations narrow the interval. A small study producing a dramatic point estimate alongside an enormous interval has established very little, however confidently the result is described in secondary coverage.
A relative change can look dramatic while the absolute change is negligible. A risk moving from two in ten thousand to one in ten thousand is a halving, and a difference of one person in ten thousand. Both are true, and a report giving only the relative figure has chosen the more impressive framing.
For ratios the interval is usually computed on a logarithmic scale, so it does not sit evenly around the point estimate. Eyeballing the midpoint of a published ratio interval will mislead, which matters because the published centre is the estimate rather than the average of the two ends.
Overlapping intervals usually mean two studies do not contradict each other, whatever their headline numbers suggest. Non-overlapping intervals are the stronger signal, and are considerably rarer than press coverage of disagreement implies.
Two intervals that do not overlap indicate a difference, but intervals can overlap substantially while the difference between the two estimates is still statistically clear. Judging by eye from overlap is therefore conservative in one direction and silent in the other, and the formal comparison is a test on the difference itself rather than an inspection of two ranges.
An interval spanning from a meaningful harm to a meaningful benefit says the study was uninformative, even where the point estimate sits near zero. An interval tightly bracketing zero says something much stronger: that a large effect has been ruled out. Both are commonly summarised as no significant difference, and they are close to opposite findings.