A hypothesis test (DfE content area O · Statistical hypothesis testing) uses data from a sample to make a decision about a population parameter. At AS level the parameter is the probability $p$ of "success" in a binomial model — the probability that a coin lands heads, that a seed germinates, that a patient recovers.
The whole topic rests on a small set of precisely defined words. Each section below defines one or two of them, with an example in the same context so you can see how they fit together.
The alternative hypothesis $H_1$ is what you will conclude is likely if the evidence goes against $H_0$. It states a range of values.
Write hypotheses for the website context, and for two further situations.
| Term | Definition | Website example |
|---|---|---|
| Test statistic | A quantity calculated from the sample, used to decide whether to reject $H_0$. | $X$ = number of the $20$ visitors who buy |
| Null distribution | The distribution of the test statistic when $H_0$ is true. | $X \sim \mathrm{B}(20,\,0.3)$ |
| Significance level $\alpha$ | The probability threshold below which a result is judged too unlikely to have happened by chance under $H_0$. | $5\%$ |
| One-tailed test | $H_1$ is $p > p_0$ or $p < p_0$; only one direction counts as evidence. | $H_1: p > 0.3$ |
| Two-tailed test | $H_1$ is $p \neq p_0$; extreme results in either direction count, so $\alpha$ is split equally between the two tails. | — |
State whether each test is one- or two-tailed and write $H_1$.
For a one-tailed test at level $\alpha$, the critical region is chosen to be as large as possible while its total probability under $H_0$ is still no more than $\alpha$.
For the website test ($H_0: p = 0.3$, $H_1: p > 0.3$, $n = 20$, $5\%$ level), find the critical region.
In the website sample, $10$ of the $20$ visitors make a purchase. Find the p-value and make a decision.
State the actual significance level of the website test, and interpret it.
A full conclusion has two parts: a statistical decision, and what it means in context. The language must be non-assertive, because a test never proves anything.
| Decision | Good conclusion |
|---|---|
| Reject $H_0$ | "There is sufficient evidence at the $5\%$ level to suggest that the proportion of visitors making a purchase has increased." |
| Do not reject $H_0$ | "There is insufficient evidence at the $5\%$ level to suggest that the proportion of visitors making a purchase has increased." |
Explain what is wrong with each conclusion.
$H_0$
Assumed true; one exact value, $p = p_0$.
$H_1$
$p > p_0$, $p < p_0$ (one-tailed) or $p \neq p_0$ (two-tailed).
Test statistic
The count $X$; under $H_0$, $X \sim \mathrm{B}(n,\,p_0)$.
Significance level
Threshold probability for rejecting $H_0$.
Critical region
Values of $X$ that lead to rejecting $H_0$; probability $\leqslant \alpha$.
Critical value
The boundary value of $X$ inside the critical region.
Acceptance region
Values of $X$ that do not lead to rejecting $H_0$.
p-value
P(observed or more extreme $\mid H_0$).
Actual significance level
P(critical region $\mid H_0$) = P(wrongly rejecting $H_0$).
Give probabilities to 4 decimal places. Use the binomial functions on your calculator.
Explain the difference between the null hypothesis and the alternative hypothesis.
▶ Show solution
The null hypothesis $H_0$ is the claim assumed to be true; it gives a single exact value of the parameter, such as $p = 0.3$.
The alternative hypothesis $H_1$ is what is suspected instead and is accepted as likely if there is enough evidence against $H_0$; it gives a range of values, such as $p > 0.3$.
A games company says that a sticker appears in $1$ in $6$ packets of cards. A collector thinks stickers are more common than this. Define a parameter and write down suitable hypotheses.
▶ Show solution
Let $p$ be the probability that a packet contains a sticker.
$H_0: p = \tfrac16$, $H_1: p > \tfrac16$
In a town, $35\%$ of adults cycle to work. A councillor wants to test whether the proportion is different in a neighbouring town. Write down hypotheses and state whether the test is one- or two-tailed.
▶ Show solution
Let $p$ be the probability that an adult in the neighbouring town cycles to work.
$H_0: p = 0.35$, $H_1: p \neq 0.35$. "Different" allows either direction, so the test is two-tailed.
A two-tailed test of $H_0: p = 0.3$ with $n = 20$ has critical region $X \leqslant 1$ or $X \geqslant 11$. (a) Write down the critical values. (b) Write down the acceptance region.
▶ Show solution
(a) The critical values are $1$ and $11$.
(b) Acceptance region: $2 \leqslant X \leqslant 10$.
For the test in Question 4, find the actual significance level.
▶ Show solution
Under $H_0$, $X \sim \mathrm{B}(20,\,0.3)$.
$\mathrm{P}(X \leqslant 1) = 0.0076$
$\mathrm{P}(X \geqslant 11) = 1 - \mathrm{P}(X \leqslant 10) = 1 - 0.9829 = 0.0171$
Actual significance level $= 0.00764 + 0.01714 = 0.0248$ (to 4 d.p.), i.e. $2.48\%$. Add the unrounded values — adding the 4 d.p. figures gives $0.0247$.
$H_0: p = 0.5$, $H_1: p > 0.5$, and the observed value of $X \sim \mathrm{B}(12,\,0.5)$ is $10$. Find the p-value and state whether the result is significant at the $5\%$ level.
▶ Show solution
p-value $= \mathrm{P}(X \geqslant 10) = 1 - \mathrm{P}(X \leqslant 9) = 1 - 0.9807 = 0.0193$.
$0.0193 < 0.05$, so the result is significant: reject $H_0$.
A student writes: "The p-value is $0.0193$, so there is a $1.93\%$ chance that $H_0$ is true." Explain the error.
▶ Show solution
The p-value is calculated assuming $H_0$ is true. It is the probability of getting a result at least as extreme as the one observed if $H_0$ is true.
It is not the probability that $H_0$ is true — $H_0$ is either true or false, and the test does not give a probability for that.
Rewrite this conclusion so that it would earn full marks: "Reject $H_0$. The dice is definitely biased." (The test was of $H_1: p > \tfrac16$, where $p$ is the probability of a six, at the $5\%$ level.)
▶ Show solution
"Reject $H_0$. There is sufficient evidence at the $5\%$ level to suggest that the die shows a six more often than one time in six."
The word "definitely" is too assertive, and "biased" does not match the one-tailed alternative hypothesis.
A test of $H_0: p = 0.2$ against $H_1: p > 0.2$ uses a sample of $25$, at the $5\%$ level. The critical region is $X \geqslant 9$. (a) Find the actual significance level. (b) The observed value is $8$. Write a conclusion.
▶ Show solution
(a) Under $H_0$, $X \sim \mathrm{B}(25,\,0.2)$. $\mathrm{P}(X \geqslant 9) = 1 - \mathrm{P}(X \leqslant 8) = 1 - 0.9532 = 0.0468$, i.e. $4.68\%$.
(b) $8$ is not in the critical region, so do not reject $H_0$. There is insufficient evidence at the $5\%$ level to suggest that $p$ is greater than $0.2$.
A café owner believes that $25\%$ of customers order a hot chocolate. A new barista thinks the proportion is higher. A random sample of $20$ customers is taken.
(a) Define a parameter and write down hypotheses. (b) Find the critical region at the $5\%$ level. (c) State the actual significance level and explain what it means here. (d) $8$ customers in the sample order hot chocolate. What should the barista conclude?
▶ Show solution
(a) Let $p$ be the probability that a customer orders a hot chocolate. $H_0: p = 0.25$, $H_1: p > 0.25$.
(b) Under $H_0$, $X \sim \mathrm{B}(20,\,0.25)$.
$\mathrm{P}(X \geqslant 8) = 1 - \mathrm{P}(X \leqslant 7) = 0.1018$ — too big.
$\mathrm{P}(X \geqslant 9) = 1 - \mathrm{P}(X \leqslant 8) = 0.0409$ — less than $0.05$ ✓
Critical region: $X \geqslant 9$.
(c) Actual significance level $= 0.0409 = 4.09\%$. If the true proportion really is $25\%$, there is a $4.09\%$ chance that the test will wrongly suggest that it is higher.
(d) $8$ is not in the critical region, so do not reject $H_0$. There is insufficient evidence at the $5\%$ level to suggest that more than $25\%$ of customers order a hot chocolate.