A single study, however well conducted, establishes that a result occurred once under one set of conditions.
A single study, however well conducted, establishes that a result occurred once under one set of conditions. Independent repetition is what converts that into a property of the world rather than a property of that study.
Systematic attempts to repeat published findings across several fields have succeeded well under half the time. The headline rates vary with definition and the direction is consistent, and it changed how carefully single studies are now read.
Replications typically find smaller effects than the originals. Publication favours striking results, and striking results are disproportionately the ones where chance ran in a helpful direction. Shrinkage on repetition is the expected consequence rather than a sign of error.
A direct replication repeats the method as closely as possible. A conceptual one tests the same idea differently. The second is more common and more publishable, and failure is easier to dismiss because the method differed.
Journals prefer novelty, so repeating someone else's work is poor career investment. The structural result is that the literature accumulates first findings and few checks, which is the condition that made the replication problem invisible for decades.
Registering a replication's analysis plan prevents it being adjusted until it agrees, and registering the original makes a faithful repeat possible. Much of the recent improvement in replication rates traces to this.
Not necessarily that the original was wrong. It means the effect is not robust to the differences between the two studies, and identifying those differences is often more informative than the verdict.
Ask whether anyone has repeated it. A claim repeated independently by unconnected groups is in a different category from one resting on its original paper, however often that paper is cited.
Giving the same dataset and question to several independent teams produces a spread of results, sometimes including opposite conclusions. These studies show that analytic choices alone carry substantial variation, independent of data collection, and they are among the more unsettling findings in recent methodology.
Some journals now accept a replication's design before results exist and commit to publishing whatever is found. This removes the publication filter from the replication itself, and findings from that format carry more weight than an ordinary replication that happened to be publishable.
The claims on this page about a finding repeated independently is a different object from a finding reported once can be checked against sources outside this site: