Reviews, testimonials and follow-up surveys collect from people who stayed. Those who stopped, left, or had a bad outcome are disproportionately absent, and the resulting picture is of the survivors rather than of everyone who started.
Reviews, testimonials and follow-up surveys collect from people who stayed. Those who stopped, left, or had a bad outcome are disproportionately absent, and the resulting picture is of the survivors rather than of everyone who started.
Nothing in the collected data indicates what is missing. A page of positive reports looks identical whether it represents everyone's experience or only the experience of those who had reason to stay, which is why the bias survives scrutiny of the data itself.
Trials and cohorts lose participants, and loss correlated with outcome distorts the result. This is why reporting how many were lost, and how they differed, is a requirement rather than a courtesy.
People with strong experiences write reviews; people with ordinary ones mostly do not. The distribution of published reviews is therefore bimodal in a way the underlying experience is not.
If who began was already unrepresentative, survivorship adds a second filter. Two filters produce a sample that can be very far from the population while each step looks defensible.
Measuring the denominator: how many started, how many finished, and what happened to the difference. Sources that publish the denominator are making a much stronger claim than sources that publish only outcomes.
Anywhere outcomes take time and people can leave — long-term use, programmes, and anything requiring persistence. The longer the horizon, the larger the gap between the survivors and the starters.
Before accepting a pattern from collected experiences, ask who would be absent from this collection, and whether their absence would change the pattern. If it would, the pattern is about the collection.
Any source reporting outcomes without reporting how many started is reporting a numerator. Asking how many began, and what happened to the difference, converts a testimonial page into something that can be assessed. Sources that publish the denominator are rare enough to be a signal by themselves.
Each period of attrition filters the group again, so the longer a programme or regimen runs, the less the remaining participants resemble those who started. Long-horizon claims drawn from remaining participants are the most distorted and usually the most confidently stated.
The claims on this page about reading only from the people still present, and what that systematically omits can be checked against sources outside this site: