Mean and variance of X̄

Two lines of algebra

A sample of $n$ observations $X_1,X_2,\dots ,X_n$ drawn independently from a population with mean $\mu$ and variance ${\sigma }^2$ has sample mean

$$\bar{X}=\frac{X_1+X_2+\cdots +X_n}{n}=\frac{1}{n}(X_1+\cdots +X_n)$$

This is a sum of independent random variables, scaled by $1/n$ — exactly Module 6.3's situation. Apply the two rules.

Means add, and a constant factor passes through:

$$E(\bar{X})=\frac{1}{n}(\mu +\mu +\cdots +\mu )=\frac{n\mu }{n}=\mu$$

Variances add for independent variables, and a constant factor is squared:

$$\mathrm{Var}(\bar{X})=\frac{1}{n^2}({\sigma }^2+{\sigma }^2+\cdots +{\sigma }^2)=\frac{n{\sigma }^2}{n^2}=\frac{{\sigma }^2}{n}$$

Taking the square root,

$$\mathrm{SD}(\bar{X})=\frac{\sigma }{\sqrt{n}}$$

The first result is L8.1's "unbiased" in symbols. The second is the formula behind L8.1's "tighter": the variance of a mean of $n$ is the population variance divided by $n$, so the SD is divided by $\sqrt{n}$ — not by $n$.

The standard error

$\sigma /\sqrt{n}$ has a name: the standard error of the mean, $\mathrm{SE}(\bar{X})$. The vocabulary matters because two different spreads are now in play:

• the standard deviation $\sigma$ describes the data — how much one invoice differs from another;
• the standard error $\sigma /\sqrt{n}$ describes the estimate — how much $\bar{x}$ would differ from one sample of $n$ to the next.

The paper penalises confusing them. "The SD of the sample mean" and "the standard error" are the same thing; "the SD of the sample" is $s$, a different thing.

Worked example — Northfield's invoices

Northfield Credit's invoice amounts have $\sigma = \$15$. For a sample of $n = 36$ invoices,

$$\mathrm{SE}(\bar{X})=\frac{15}{\sqrt{36}}=\frac{15}{6}=\$2.50$$

One invoice varies from the mean by about \$15; the average of 36 invoices varies by about \$2.50. Dividing by 36 instead of $\sqrt{36}$ gives \$0.42 — a spread six times too small, and the most common error on this formula.

How large a sample halves the SE to \$1.25? Solve $15/\sqrt{n}=1.25$: $\sqrt{n}=12$, $n=144$. Halving the standard error needs four times the sample, because $n$ sits under a square root. In general, $n=(\sigma /\mathrm{SE}{)}^2$ — the formula Module 9's minimum sample size builds on.

The law of large numbers

As $n$ grows, $\sigma /\sqrt{n}$ shrinks toward zero: the sample mean concentrates ever more tightly around $\mu$. That is the law of large numbers — with enough data, $\bar{x}$ is as close to $\mu$ as you like — and it is why a large sample is worth paying for, at the diminishing rate the square root imposes. What the two rules do not say is anything about the shape of $\bar{X}$'s distribution. That takes L8.3 and L8.4.