The bibliography is evidence about the paper, before you read a word of it

A paper citing nothing from the last five years is either reviewing settled history or has not engaged with current work.

Age distribution is informative

A paper citing nothing from the last five years is either reviewing settled history or has not engaged with current work. A paper citing almost nothing older than five years may be missing the foundational results its claims rest on.

Self-citation has a normal range

Authors cite their own prior work legitimately, because research is cumulative. A reference list where a large share points at one group is a signal to check whether the claim is supported outside that group.

Check whether the citation says what it is used for

The most common citation error is a reference that does not support the sentence attached to it. Following two or three citations in any paper is a fast and unusually revealing check, and it frequently finds a claim resting on a source that said something weaker.

Secondary citation propagates error

Authors cite a paper for a claim they read in another paper's description of it. A misdescription can then travel through a literature for years, gaining apparent authority with each repetition while nobody rereads the original.

Review-heavy lists are weaker

A list made mostly of reviews rather than primary studies means the authors are reporting consensus rather than evidence. That is appropriate for background and not for the central claim of a paper.

Look for the inconvenient citations

A paper engaging with work that disagrees with it is doing something a paper citing only supportive sources is not. Absence of any contrary citation in a contested area is itself a finding about the paper.

Predatory venues appear in lists

Citations to journals with no visible review process weaken the chain regardless of the citing paper's quality. Checking two or three unfamiliar journal names takes a minute.

What to do with this

Scan the list before the body. Three questions — how recent, how self-contained, and does it engage with disagreement — place a paper faster than reading its introduction.

Follow two citations at random

The most common citation error is a reference that does not support the sentence attached to it. Checking two takes a few minutes and is unusually diagnostic of how carefully a paper was assembled.

Look for the inconvenient citation

A paper engaging with work that disagrees with it is doing something a paper citing only allies is not. In a contested area, no contrary citation at all is itself a finding.

Where to check this independently

The claims on this page about the bibliography is evidence about the paper, before you read a word of it can be checked against sources outside this site:

HEALTHYLIFE10 gives 10% off sitewide at growthguys.com. No minimum, no exclusions, no account. If you want the evidence rather than the claim, it is at the evidence index and on the full code page.

Related