p-value

The p-value is the probability, computed assuming the null hypothesis and the test's model are true, of getting a test statistic at least as extreme as the one observed.

p_{\text{greater}} = P(T \ge t_{\text{obs}} \given H_0), \qquad p_{\text{less}} = P(T \le t_{\text{obs}} \given H_0)

One-sided p-values are single tail areas of the null distribution.

$T$: The test statistic as a random variable under $H_0$ (e.g. t, z, F, χ²).
$t_{\text{obs}}$: The value of the statistic computed from the data.

p_{\text{two-sided}} = \min\{1,\ 2\min(p_{\text{less}},\, p_{\text{greater}})\}

Engine convention: double the smaller tail (exact binomial and Fisher tests instead sum the probabilities of all outcomes no more likely than the observed one).

$p_{\text{less}}, p_{\text{greater}}$: The lower- and upper-tail probabilities of the observed statistic.

When to use

Report a p-value with any hypothesis test, together with the estimate, a confidence interval and an effect size. Chi-square and ANOVA F tests are naturally upper-tailed: their p-value is the upper tail only.

Assumptions

The p-value is computed assuming $H_0$ is true and the test's model holds (independence, sampling design, distributional conditions). If the model is wrong, the p-value is wrong.

How to interpret

A small p-value means: if $H_0$ were true, data this extreme would be rare — so the data are evidence against $H_0$ .
A large p-value means the data are consistent with $H_0$ — not that $H_0$ is true.
p is a continuous measure of evidence; $\alpha$ turns it into a decision. Report the exact value (e.g. $p = 0.031$ ), not just "p < 0.05".

Common mistakes

These readings are all wrong:

"p is the probability that $H_0$ is true." (It is computed assuming $H_0$ is true.)
"1 − p is the probability that $H_1$ is true."
"p is the probability that the result happened by chance."
"p > 0.05 shows there is no effect." (Absence of evidence is not evidence of absence; check power and the interval.)
"A smaller p means a bigger or more important effect." (p also shrinks with sample size.)
"p = 0.049 and p = 0.051 are fundamentally different results."
Doubling a one-sided p-value after the fact, or choosing the side after seeing the data.

Example

For the eight delivery times (mean 13.5 min, $s = 1.773$ ) tested against $\mu_0 = 12$ : $t = 2.39$ on 7 df. The upper tail $P(T \ge 2.39) = 0.024$ , so the two-sided p-value is $2 \times 0.024 = 0.048$ . Reading: if the true mean were 12 minutes, a sample mean at least this far from 12 (in either direction) would occur in about 4.8% of samples of 8.