Hand versus R: the three mismatches

◈ 6 cards

Every gap between a hand answer and a printout in this course was one of three things — table rounding (1.645 vs 1.644854), the Welch df convention (9 vs 19.62), or the continuity correction in prop.test (0.0784 vs 0.0681) — and each has a stated tolerance.

Three reasons, and only three

The paper prints R output and expects a hand answer beside it. They rarely match to the last digit, and the question that follows — explain the difference — is a mark. In this course every difference had one of three causes.

1. Table rounding

The tables carry three or four decimals; R carries seven. $z_{0.05} = 1.645$ on the sheet is 1.644854 in R; $Φ (1.49) = 0.9319$ is 0.9318879. A confidence bound computed with 1.645 differs from R’s by a few parts in ten thousand, and a $p$ -value read after rounding $z$ to two decimals differs in the fourth place.

> qnorm(0.95); qt(0.95, 9); qt(0.95, 19.62)
[1] 1.644854
[1] 1.833113
[1] 1.72634

The first two lines are the table’s 1.645 and 1.833 unrounded. Tolerance: the course grades a $z$ or $t$ to ±0.01, a table $p$ to ±0.002 when you rounded $z$ first, a CI bound to ±0.5 % of its value — all of which absorb table rounding. A hand bound 0.3 % wider than R’s is not an error.

2. The df convention

The third line above is the Welch case. L11.3’s samples ( $n = 12$ and $10$ ) use df $= min (11, 9) = 9$ by hand and R’s Satterthwaite $19.62$ . The critical value drops from 1.833 to 1.726, and the $p$ -value for the same $t = 2.037$ :

> pt(2.037, 9, lower.tail = FALSE); pt(2.037, 19.62, lower.tail = FALSE)
[1] 0.036057
[1] 0.02768478

Hand: $0.025 < p < 0.05$ (bracketed on row 9); R: 0.0277. Both reject at 5 %; R’s $p$ is smaller because its df is larger — the minimum rule is deliberately conservative. This is not rounding and no tolerance closes it; the answer is a sentence: R uses the Satterthwaite df, 19.62; by hand the course uses the minimum rule, 9, which gives a larger critical value and a larger $p$ . The ratio $1.726/1.833 = 0.942$ says how much the convention costs: about 6 % on the critical value here.

3. The continuity correction

prop.test applies a ±0.5 correction by default; the hand $z$ -test does not.

> prop.test(60, 500, 0.1, "greater")$p.value; prop.test(60, 500, 0.1, "greater", correct = FALSE)$p.value
[1] 0.07836193
[1] 0.06801856

The second line is the hand $p = 0.0681$ (L12.3) to table precision; the first is the corrected version, larger by a hundredth. When a prop.test title says with continuity correction, expect its $p$ to sit above yours; correct = FALSE reproduces the hand answer. The gap can move a borderline decision, and the sentence is: R’s default applies a continuity correction to the count; the hand test does not.

Reading a mismatch on the paper

Find the size of the gap. In the fourth decimal of a $z$ , $t$ or CI bound → rounding, within tolerance, say so and move on. A different df printed beside t = → the convention; name both df. A prop.test with continuity correction in its title → the correction; name it. Anything else — a different tail, a different $α$ , an SE built from $\overset{p}{^}$ where $p_{0}$ belonged — is a mistake, yours or the question’s, and worth finding before you write "rounding".