p-values

◈ 7 cards

The p-value is the probability, if H₀ were true, of a statistic at least as extreme as the one observed — one tail for a one-sided test, doubled for two-sided; reject when p ≤ α.

The second route to the same decision

L10.3 compared the statistic with a critical value. The p-value route compares a probability with $α$ instead:

> The p-value is the probability, computed assuming $H_{0}$ is true, of getting a test statistic at least as extreme — in the direction of $H_{a}$ — as the one observed.

A small p-value says: if the null were true, a sample like this would be rare. Rare enough, and the null goes. The decision rule is

$p \leq α; ⟹; reject H_{0}, p > α; ⟹; do not reject$

The two routes always agree — the p-value is the smallest $α$ at which the observed statistic would reject — but the p-value carries more information: it says how rare, not just whether it crossed a line. That is why software prints it.

Worked example — the same z, three p-values

Kitchener Ridge's lower-tailed test gave $z = - 1.90$ . "At least as extreme in the direction of $H_{a}$ " means this far below, or further:

$p = P (Z \leq - 1.90) = Φ (- 1.90) = 1 - Φ (1.90) = 1 - 0.9713 = 0.0287$

$0.0287 \leq 0.05$ : reject at the 5 % level — the same decision as the critical-value route, as it must be. Note also $0.0287 > 0.01$ : at the 1 % level the same data would not reject. The p-value lets the reader apply their own $α$ .

Two-sided. If the question had been has the mean changed?, "at least as extreme" means this far from 45 in either direction, so both tails count:

$p = 2 \times Φ (- 1.90) = 2 \times 0.0287 = 0.0574$

$0.0574 > 0.05$ : do not reject at 5 % — again matching L10.3's $∣ - 1.90 ∣ < 1.96$ . The two-sided p is double the one-sided p, always.

A second statistic. A different test yields $z = 2.31$ , two-sided:

$p = 2 \times (1 - Φ (2.31)) = 2 \times (1 - 0.9896) = 2 \times 0.0104 = 0.0209$

Reject at 5 %, not at 1 %. The recipe: find the one-tail area beyond $∣ z ∣$ from the cumulative table, $1 - Φ (∣ z ∣)$ ; use it as is for a one-sided test whose direction the data agree with; double it for two-sided.

$z$	one-sided $p$	two-sided $p$	at $α = 0.05$
$- 1.90$	0.0287	0.0574	reject / do not reject
$2.31$	0.0104	0.0209	reject / reject

What a p-value is not

$p = 0.0287$ does not mean there is a 2.87 % chance that $H_{0}$ is true. $H_{0}$ is true or it is not; the 2.87 % is the chance of data this extreme if it is. Nor does $p$ measure the size of the effect: a tiny difference with a huge $n$ can have a tiny $p$ (L10.6). Read it as a rarity, under the null, of what was seen.

Software prints two-sided

R's t.test and every other package print the two-sided p-value by default. If your question is one-sided and the estimate lies in the direction $H_{a}$ predicts, halve the printed value; if the estimate lies the wrong way, the one-sided p is greater than 0.5 and the null stands. L10.8 shows the alternative = argument that makes R do the halving itself.

One-sided p is the single tail beyond |z|; two-sided doubles it. The same z = −1.90 rejects one-sided and fails two-sided at 5 %.