Hypothesis tests

6.5 Hypothesis tests

The language of testing

A hypothesis test uses a sample to decide whether a claim about a population is believable.

One-tailed and two-tailed tests

For a two-tailed test, half the significance level goes in each tail.

Testing a binomial proportion

Work out the probability of the observed result, or one more extreme, assuming $\mathrm{H}_0$.

$\mathrm{H}_0$: $p = 0.5$. $\mathrm{H}_1$: $p > 0.5$. Under $\mathrm{H}_0$, $X \sim \mathrm{B}(20, 0.5)$.

$\mathrm{P}(X \geq 15) = \frac{15504 + 4845 + 1140 + 190 + 20 + 1}{2^{20}} = 0.0207$

$0.0207 < 0.05$, so reject $\mathrm{H}_0$. There is evidence that the coin is biased towards heads.

Finding a rejection region

Find the most extreme values whose total probability is no more than the significance level.

$\mathrm{P}(X \geq 15) = 0.0207 < 0.05$, while $\mathrm{P}(X \geq 14) = 0.0577 > 0.05$. So the rejection region is $X \geq 15$.

Testing a Poisson mean

$\mathrm{H}_0$: $\lambda = 6$. $\mathrm{H}_1$: $\lambda < 6$. Under $\mathrm{H}_0$, $X \sim \mathrm{Po}(6)$.

$\mathrm{P}(X \leq 2) = \mathrm{e}^{-6}\left(1 + 6 + 18\right) = 0.0620$

$0.0620 > 0.05$, so do not reject $\mathrm{H}_0$. There is not enough evidence that the rate of complaints has fallen.

Using normal approximations

When $n$ or $\lambda$ is large, use the normal, and remember the continuity correction.

$\mathrm{H}_0$: $p = 0.4$. $\mathrm{H}_1$: $p > 0.4$. Under $\mathrm{H}_0$, $X \sim \mathrm{B}(100, 0.4)$, approximated by $\mathrm{N}(40, 24)$.

$z = \frac{49.5 - 40}{\sqrt{24}} = 1.939$

$1.939 > 1.645$, so reject $\mathrm{H}_0$. There is evidence that the proportion has increased.

Testing a population mean

Use the distribution of the sample mean. This needs a normal population with known variance, or a large sample.

$\mathrm{H}_0$: $\mu = 50$. $\mathrm{H}_1$: $\mu \neq 50$. Under $\mathrm{H}_0$, $\bar{X} \sim \mathrm{N}(50, 0.64)$.

$z = \frac{48.2 - 50}{0.8} = -2.25$

$\lvert -2.25\rvert > 1.96$, so reject $\mathrm{H}_0$. There is evidence that the mean mass has changed.

Type I and Type II errors

Probability of a Type I error. For a normal test, this is the significance level. For a binomial or Poisson test, it is the actual probability of landing in the rejection region under $\mathrm{H}_0$, which is usually below the stated level. In the coin test, it is $0.0207$.

Probability of a Type II error. Find the rejection region, then work out the probability of falling outside it, using the true value.

The rejection region is $\bar{x} > 50 + 1.645 \times 0.8 = 51.316$.

A Type II error means $\bar{X} < 51.316$ when $\mu = 52$:

$z = \frac{51.316 - 52}{0.8} = -0.855 \qquad \mathrm{P} = 1 - \Phi(0.855) = 0.196$