The sampling distribution of a difference

Means subtract, variances add

Two independent samples give two sample means. Module 8 said each one is approximately normal: ${\bar{X}}_1\sim N({\mu }_1,{\sigma }_1^2/n_1)$ and ${\bar{X}}_2\sim N({\mu }_2,{\sigma }_2^2/n_2)$. Module 6’s rules for combining independent random variables then give the distribution of the difference:

$$E({\bar{X}}_1-{\bar{X}}_2)={\mu }_1-{\mu }_2,\mathrm{Var}({\bar{X}}_1-{\bar{X}}_2)=\frac{{\sigma }_1^2}{n_1}+\frac{{\sigma }_2^2}{n_2}$$

The expected value subtracts, as intuition says. The variance adds — subtracting an independent quantity adds its uncertainty, exactly as $\mathrm{Var}(X-Y)=\mathrm{Var}(X)+\mathrm{Var}(Y)$ in Module 6. The standard error of the difference is the square root:

$$\mathrm{SE}({\bar{X}}_1-{\bar{X}}_2)=\sqrt{\frac{{\sigma }_1^2}{n_1}+\frac{{\sigma }_2^2}{n_2}}$$

Each variance is divided by its own sample size before the two are added. The SDs are never added and never subtracted.

Worked example — two offices, σ known

Maple Ledger has long-run records of days-to-pay in two offices: Kitchener has ${\sigma }_1=6$ days, Guelph ${\sigma }_2=8$ days. Fresh samples of $n_1=36$ and $n_2=64$ invoices give ${\bar{x}}_1-{\bar{x}}_2=3.5$ days. Is that difference more than sampling noise?

$$\mathrm{SE}=\sqrt{\frac{36}{36}+\frac{64}{64}}=\sqrt{1+1}=1.414$$

Under $H_0:{\mu }_1-{\mu }_2=0$ the observed difference is standardised as in Module 10:

$$z=\frac{({\bar{x}}_1-{\bar{x}}_2)-0}{\mathrm{SE}}=\frac{3.5}{1.414}=2.47$$

The difference sits 2.47 standard errors above zero. Two-sided at $\alpha =0.05$ the critical values are $\pm 1.96$: reject. Both samples are over 30, so the CLT covers the normal shape whatever the invoice-level distribution looks like.

A note on the arithmetic trap: ${\sigma }_1/\sqrt{n_1}-{\sigma }_2/\sqrt{n_2}=1-1=0$ would make the SE zero and the statistic infinite — nonsense that comes from subtracting SEs. Only variances combine.

When z applies to two means

The z version needs both population SDs given. That is rare in practice and appears on the paper mainly to establish the SE formula; from L11.3 the population SDs are unknown, $s_1$ and $s_2$ replace them, and the statistic becomes t. The SE formula does not change — only what goes into it and which table is read.

The general null value

The hypothesis need not be no difference. If Kitchener is expected to be 2 days faster, $H_0:{\mu }_1-{\mu }_2=2$ and the numerator becomes $({\bar{x}}_1-{\bar{x}}_2)-2$. Write the null value ${\delta }_0$ in the general form

$$z=\frac{({\bar{x}}_1-{\bar{x}}_2)-{\delta }_0}{\mathrm{SE}}$$

and set ${\delta }_0=0$ when the question asks simply whether the means differ, which is nearly always.