The model assumptions and the residual standard error

From a fitted line to a population model

Module 4 fitted $\hat{y}=544.18+2.066x$ to eight Northfield Credit clients and stopped: it described the eight. Inference asks about all clients — is the population slope ${\beta }_1$ different from zero, and what range of values is plausible for it? That needs a model of how the eight points came to be:

$$Y={\beta }_0+{\beta }_{1}X+\epsilon ,\epsilon \sim N(0,{\sigma }^2)\text{ independent}$$

The population line is straight; each observed score is its point on that line plus an error $\epsilon$, and the errors behave like a normal random sample with one common spread $\sigma$. Unpacked, that is four assumptions, each buying something:

1. Linearity — the mean of $Y$ at each $x$ lies on a straight line. Buys: the slope means the same thing everywhere in the range.
2. Independence — one client’s error tells you nothing about another’s. Buys: the standard errors of Modules 8–12 apply.
3. Constant variance — $\sigma$ is the same at every $x$ (homoscedasticity). Buys: one $s$ describes the scatter, so one SE describes the slope.
4. Normal errors — $\epsilon \sim N(0,{\sigma }^2)$. Buys: the $t$ distribution for the slope with small $n$.

L13.4 shows how residual plots check 1, 3 and 4. Assumption 2 is the one no plot rescues after the fact — it is a property of how the data were collected, exactly Module 2’s sampling questions.

Estimating σ — the residual standard error

The errors $\epsilon$ are unobservable; the residuals $e=y-\hat{y}$ are their stand-ins. Their spread estimates $\sigma$:

$$s=\sqrt{\frac{SSE}{n-2}}=\sqrt{\frac{\sum e^2}{n-2}}$$

The divisor is $n-2$, not $n-1$: two parameters (${\hat{\beta }}_0$, ${\hat{\beta }}_1$) were estimated from the data before the residuals could be computed, and each one costs a degree of freedom. Module 3’s $s$ lost one to $\bar{x}$; regression loses two to the line.

Worked example — Northfield’s s

Module 4 found the eight residuals −6.5, 1.7, −11.6, 9.8, 11.2, 2.3, 4.6, −11.4 and $SSE=557.8$.

$$s=\sqrt{\frac{557.8}{8-2}}=\sqrt{92.97}=9.642\text{on }6\text{ degrees of freedom}$$

Read it as the typical size of a residual: a client’s actual score sits about 9.6 points from the fitted line, on average. In L13.2 it is the numerator of every standard error; in L13.4 twice it is the outlier threshold; in R it is the line Residual standard error: 9.642 on 6 degrees of freedom that Module 4 told you to read as description.

What s is not

It is not $s_y=53.3$, the spread of scores around their mean — that ignores income entirely. It is the spread around the line, and the fact that 9.6 is so much smaller than 53.3 is another way of saying $R^2=0.972$: the line has absorbed most of the variation. And it is not $R^2$: $s$ is in the units of $y$ (points), $R^2$ is a share.