A p-value is the probability of observing a result at least this extreme if there were genuinely no effect.
A p-value is the probability of observing a result at least this extreme if there were genuinely no effect. It is a statement about data under an assumption, and nothing more than that. Everything else attributed to it in secondary coverage is an addition rather than a reading.
It is not the probability that a hypothesis is true, not the probability the result was a fluke, and not a measure of how large or important an effect is. All three readings appear routinely in reporting, and all three are wrong in ways that change what a reader concludes.
The threshold is a convention rather than a property of nature. A result at 0.049 and one at 0.051 represent near-identical evidence, and treating one as a finding and the other as nothing is a publishing habit. Several fields have moved away from the convention for exactly this reason.
With a large enough sample, trivially small differences become statistically significant. A significant result attached to a negligible effect size is a precise measurement of something that does not matter, and the significance figure alone cannot tell the two situations apart.
Testing many outcomes raises the chance that one clears the threshold by luck alone. A study that measured twenty endpoints and reports the one that reached significance is describing the expected behaviour of randomness rather than a discovery, unless it corrected for the number of tests and says so.
Most of the ways a p-value misleads come from decisions made after seeing data: which outcome to report, when to stop collecting, which subgroup to highlight. Registering the analysis plan beforehand removes that freedom, and the gap between registered and reported outcomes is now itself a studied phenomenon.
The p-value compresses evidence into a yes-or-no against an arbitrary threshold. An interval shows the whole range the data support, so a reader can see whether a non-significant result rules out a meaningful effect or merely failed to detect one. Those are very different conclusions.
How large was the effect, how precisely was it measured, and was this the outcome the study said it would measure before it began. Those three questions recover almost everything the single figure discards.
The figure depends on the test chosen, and more than one test is usually defensible for a given dataset. Parametric and non-parametric alternatives make different assumptions about distribution and can land either side of a threshold. A paper naming its test and why it was appropriate is doing something a paper reporting only the result is not, and where the choice looks unusual it is worth asking whether other tests were run first.
A two-sided test asks whether there is a difference in either direction; a one-sided test asks only about the direction the researchers expected, and reaches significance more easily. One-sided tests are legitimate where the opposite direction would be meaningless, and they should be declared in advance. Undeclared, they are one of the simpler ways to halve a p-value without collecting any more data.
The claims on this page about the most cited statistic in research is also the most widely misdescribed can be checked against sources outside this site: