The least-squares line

The line that makes the errors smallest

Many lines pass near the eight points. The least-squares line $\hat{y}={\hat{\beta }}_0+{\hat{\beta }}_{1}x$ is the one that minimises the sum of the squared vertical gaps

$$\sum e_i^2=\sum (y_i-{\hat{y}}_i{)}^2$$

Vertical, because $y$ is the response being predicted; squared, so that positive and negative misses do not cancel and large misses are penalised. The minimising slope and intercept have closed forms:

$${\hat{\beta }}_1=\frac{S{P}_{xy}}{S{S}_{xx}},{\hat{\beta }}_0=\bar{y}-{\hat{\beta }}_1\bar{x}$$

where $S{S}_{xx}=\sum x^2-n{\bar{x}}^2$ and $S{P}_{xy}=\sum xy-n\bar{x}\bar{y}$ — the two numerators you already built in L4.1 and L4.2. The intercept formula says the line always passes through the point of means $(\bar{x},\bar{y})$.

Worked example — fitting Northfield's line

$S{S}_{xx}=43,035-8\times 69.375^2=43,035-38,503.125=4,531.875$.

$S{P}_{xy}=390,925-8\times 69.375\times 687.5=9,362.5$.

$${\hat{\beta }}_1=\frac{9,362.5}{4,531.875}=2.0659,{\hat{\beta }}_0=687.5-2.0659\times 69.375=687.5-143.32=544.18$$

$$\widehat{\text{score}}=544.18+2.066\times \text{income}$$

Three forms of the same slope

Because $S{S}_{xx}=(n-1)s_x^2$ and $S{P}_{xy}=(n-1)s_{xy}$, the slope can be written three ways, and the paper may hand you the inputs for any of them:

$${\hat{\beta }}_1=\frac{S{P}_{xy}}{S{S}_{xx}}=\frac{s_{xy}}{s_x^2}=r\frac{s_y}{s_x}$$

Check: $1,337.5/647.41=2.0659$; $0.98589\times 53.319/25.444=2.0659$. Same line.

The rounding trap

The third form is where marks go missing. Use $r=0.986$ (three decimals) and the slope comes out $2.0662$; use $r=0.99$ and it is $2.0745$ — off in the second decimal. The intercept then inherits the error multiplied by $\bar{x}=69.375$. Carry $r$ and the SDs to at least four decimals until the final line, or — better — take the slope from $S{P}_{xy}/S{S}_{xx}$, where every input is an exact sum.

What the fit cannot tell you

Least squares always produces a line, for any cloud of points, however curved or however dominated by one point. That the arithmetic ran is not evidence that a line is the right description; the scatter plot and $r$ have to have earned it first.