Binomial Hypothesis Tests – A Level Maths Year 1

1 The Procedure

This page puts the vocabulary from The Language of Hypothesis Testing to work (DfE content area O · Statistical hypothesis testing). Every test here is a test for a proportion $p$, using a binomial model from The Binomial Distribution.

A binomial test is appropriate when the data are a count of successes in a fixed number $n$ of independent trials, each with the same probability $p$ of success. Under $H_0$ the number of successes is $X \sim \mathrm{B}(n,\,p_0)$.

Define $p$ in words, and write $H_0: p = p_0$ and $H_1$.
State the null distribution: "Under $H_0$, $X \sim \mathrm{B}(n,\,p_0)$".
Either find the p-value of the observed result, or find the critical region.
Compare: p-value with $\alpha$ (or $\tfrac{\alpha}{2}$ for two tails); or check whether the observed value is in the critical region.
Write the decision ("reject $H_0$" / "do not reject $H_0$") and a conclusion in context.

Calculator facts

$\mathrm{P}(X \leqslant k)$ is the cumulative function directly
$\mathrm{P}(X \geqslant k) = 1 - \mathrm{P}(X \leqslant k - 1)$

Calculators are allowed and expected. Most have a "binomial CD" function giving $\mathrm{P}(X \leqslant k)$. Some also allow a lower and upper bound, which gives $\mathrm{P}(X \geqslant k)$ directly by setting the upper bound to $n$.

2 One-Tailed Tests Using a p-value

The quickest way to carry out a test when you have the observed value: calculate the probability of that value or anything more extreme in the direction of $H_1$.

Worked Example 1 — Upper tail

A coin is suspected of being biased towards heads. In $20$ throws it lands heads $15$ times. Test at the $5\%$ level.

①Let $p$ = P(heads). $H_0: p = 0.5$, $H_1: p > 0.5$.

②Under $H_0$, $X \sim \mathrm{B}(20,\,0.5)$, where $X$ is the number of heads.

③p-value $= \mathrm{P}(X \geqslant 15) = 1 - \mathrm{P}(X \leqslant 14) = 1 - 0.9793 = 0.0207$.

④$0.0207 < 0.05$: reject $H_0$.

⑤There is sufficient evidence at the $5\%$ level to suggest that the coin is biased towards heads.

Worked Example 2 — Lower tail

A manufacturer claims that $60\%$ of patients recover within a week using its treatment. A doctor suspects the figure is lower. Of $25$ patients treated, $10$ recover within a week. Test at the $5\%$ level.

①Let $p$ = P(a patient recovers within a week). $H_0: p = 0.6$, $H_1: p < 0.6$.

②Under $H_0$, $X \sim \mathrm{B}(25,\,0.6)$.

③p-value $= \mathrm{P}(X \leqslant 10) = 0.0344$.

④$0.0344 < 0.05$: reject $H_0$.

⑤There is sufficient evidence at the $5\%$ level to suggest that the recovery rate is less than $60\%$.

Worked Example 3 — A result that is not significant

Historically $10\%$ of a firm's job applicants are invited to interview. After a change to the advert, $1$ of the next $50$ applicants is invited. Test at the $1\%$ level whether the proportion has fallen.

①Let $p$ = P(an applicant is invited). $H_0: p = 0.1$, $H_1: p < 0.1$.

②Under $H_0$, $X \sim \mathrm{B}(50,\,0.1)$.

③p-value $= \mathrm{P}(X \leqslant 1) = 0.0338$.

④$0.0338 > 0.01$: do not reject $H_0$.

⑤There is insufficient evidence at the $1\%$ level to suggest that the proportion invited to interview has fallen.

At the $5\%$ level the same data would be significant. The significance level must be fixed before the data are collected — you may not shop around for one that gives the answer you want.

3 Finding a One-Tailed Critical Region

Questions often ask for the critical region before the data are known. You then compare the observed value with it.

Decide which tail: $H_1: p < p_0$ uses small values of $X$; $H_1: p > p_0$ uses large values.
Work out the cumulative probabilities near the edge of that tail.
Choose the largest region whose probability is still $\leqslant \alpha$.
Quote the probability just inside and just outside the boundary as evidence.

Worked Example 4 — Lower-tail critical region

$X \sim \mathrm{B}(30,\,p)$. Find the critical region for a test of $H_0: p = 0.4$ against $H_1: p < 0.4$ at the $5\%$ level, and the actual significance level.

①Under $H_0$, $X \sim \mathrm{B}(30,\,0.4)$; mean $12$. Look at small values.

②$\mathrm{P}(X \leqslant 7) = 0.0435$ ($\leqslant 0.05$ ✓)

③$\mathrm{P}(X \leqslant 8) = 0.0940$ ($> 0.05$ ✗)

④Critical region $X \leqslant 7$; actual significance level $4.35\%$.

Worked Example 5 — Upper-tail critical region

$35\%$ of households in a region own a dog. A vet thinks the proportion is higher in her town and plans to survey $25$ households. Find the critical region at the $5\%$ level.

①$H_0: p = 0.35$, $H_1: p > 0.35$. Under $H_0$, $X \sim \mathrm{B}(25,\,0.35)$; mean $8.75$.

②$\mathrm{P}(X \geqslant 13) = 1 - \mathrm{P}(X \leqslant 12) = 0.0604$ ($> 0.05$ ✗)

③$\mathrm{P}(X \geqslant 14) = 1 - \mathrm{P}(X \leqslant 13) = 0.0255$ ($\leqslant 0.05$ ✓)

④Critical region $X \geqslant 14$; actual significance level $2.55\%$.

The actual significance level ($2.55\%$) is well below $5\%$ here. Because $X$ jumps in whole numbers, you cannot get closer to $5\%$ without going over it.

4 Two-Tailed Tests

For $H_1: p \neq p_0$, evidence can be in either tail. The significance level is split equally: at the $5\%$ level each tail may have probability at most $2.5\%$. The critical region has two parts.

[0.0, 0.0005, 0.0031, 0.0123, 0.035, 0.0746, 0.1244, 0.1659, 0.1797, 0.1597, 0.1171, 0.071, 0.0355, 0.0146, 0.0049, 0.0013, 0.0003, 0.0, 0.0, 0.0, 0.0]

Worked Example 6 — Two-tailed critical region

$X \sim \mathrm{B}(20,\,p)$. Find the critical region for a test of $H_0: p = 0.4$ against $H_1: p \neq 0.4$ at the $5\%$ level, and the actual significance level.

①Each tail: probability $\leqslant 0.025$.

②Lower: $\mathrm{P}(X \leqslant 3) = 0.0160$ ✓, $\mathrm{P}(X \leqslant 4) = 0.0510$ ✗ $\Rightarrow X \leqslant 3$.

③Upper: $\mathrm{P}(X \geqslant 13) = 1 - \mathrm{P}(X \leqslant 12) = 0.0210$ ✓, $\mathrm{P}(X \geqslant 12) = 0.0565$ ✗ $\Rightarrow X \geqslant 13$.

④Critical region: $X \leqslant 3$ or $X \geqslant 13$.

⑤Actual significance level $= 0.0160 + 0.0210 = 0.0370 = 3.70\%$.

To carry out a two-tailed test with a p-value, first decide which tail the observation is in by comparing it with the mean $np_0$, then compare the one-tail probability with $\tfrac{\alpha}{2}$.

Worked Example 7 — Two-tailed test using a p-value

A spinner is designed to land on blue with probability $0.25$. In $40$ spins it lands on blue $16$ times. Test, at (a) the $10\%$ level and (b) the $5\%$ level, whether the probability of blue is different from $0.25$.

①$H_0: p = 0.25$, $H_1: p \neq 0.25$. Under $H_0$, $X \sim \mathrm{B}(40,\,0.25)$; mean $10$.

②$16 > 10$, so use the upper tail: $\mathrm{P}(X \geqslant 16) = 1 - \mathrm{P}(X \leqslant 15) = 0.0262$.

③(a) $10\%$ two-tailed $\Rightarrow$ compare with $0.05$: $0.0262 < 0.05$, reject $H_0$. There is sufficient evidence at the $10\%$ level to suggest that the probability of blue is not $0.25$.

④(b) $5\%$ two-tailed $\Rightarrow$ compare with $0.025$: $0.0262 > 0.025$, do not reject $H_0$. There is insufficient evidence at the $5\%$ level to suggest that the probability of blue is not $0.25$.

An alternative, equally valid, approach is to double the one-tail probability ($2 \times 0.0262 = 0.0525$) and compare with the full $\alpha$. Either way, say clearly what you are comparing.

5 Pitfalls and Model Assumptions

Off-by-one in the upper tail. $\mathrm{P}(X \geqslant 14) = 1 - \mathrm{P}(X \leqslant 13)$. Using $1 - \mathrm{P}(X \leqslant 14)$ gives $\mathrm{P}(X \geqslant 15)$ — the wrong answer.

Using the closest probability rather than the largest that does not exceed $\alpha$. If $\mathrm{P}(X \geqslant 9) = 0.0468$ and $\mathrm{P}(X \geqslant 8) = 0.0510$ were the two candidates, $0.0510$ is closer to $5\%$ but is over it, so the critical region is $X \geqslant 9$.

Stating the critical region as a probability. The critical region is a set of values of $X$, such as $X \leqslant 7$. Its probability is the actual significance level.

Is the binomial model reasonable? Questions sometimes ask you to criticise the model. Look for: a fixed number of trials; two outcomes; trials independent (e.g. patients not related, seeds not in the same tray affected together); probability the same each trial (e.g. a coin not getting worn). The sample should also be random if the conclusion is to apply to the whole population.

6 Quick Reference

Null distribution

$X \sim \mathrm{B}(n,\,p_0)$ under $H_0$.

Upper-tail p-value

$\mathrm{P}(X \geqslant x) = 1 - \mathrm{P}(X \leqslant x-1)$.

Lower-tail p-value

$\mathrm{P}(X \leqslant x)$ directly.

Decision

p-value $\leqslant \alpha$ (one tail) or $\leqslant \tfrac{\alpha}{2}$ (two tails) $\Rightarrow$ reject $H_0$.

Critical region

Largest tail with probability $\leqslant \alpha$; show values either side of the boundary.

Two-tailed region

$\tfrac{\alpha}{2}$ in each tail; actual level = sum of both tails.

Which tail?

Observed $> np_0$: upper; observed $< np_0$: lower.

Conclusion

Decision + "sufficient / insufficient evidence to suggest…" in context.

7 Practice Questions

Give probabilities to 4 decimal places. In every test, define $p$, state the hypotheses and the distribution, and conclude in context.

Question 1

$X \sim \mathrm{B}(12,\,p)$. Test $H_0: p = 0.5$ against $H_1: p > 0.5$ at the $5\%$ level, given that $X = 10$.

▶ Show solution

Under $H_0$, $X \sim \mathrm{B}(12,\,0.5)$.

$\mathrm{P}(X \geqslant 10) = 1 - \mathrm{P}(X \leqslant 9) = 1 - 0.9807 = 0.0193$.

$0.0193 < 0.05$: reject $H_0$. There is sufficient evidence at the $5\%$ level to suggest that $p > 0.5$.

Question 2

A garage says $20\%$ of cars fail their safety check. A mechanic believes more fail. Of $15$ randomly chosen cars, $6$ fail. Test at the $5\%$ level.

▶ Show solution

Let $p$ = P(a car fails). $H_0: p = 0.2$, $H_1: p > 0.2$. Under $H_0$, $X \sim \mathrm{B}(15,\,0.2)$.

$\mathrm{P}(X \geqslant 6) = 1 - \mathrm{P}(X \leqslant 5) = 1 - 0.9389 = 0.0611$.

$0.0611 > 0.05$: do not reject $H_0$. There is insufficient evidence at the $5\%$ level to suggest that more than $20\%$ of cars fail.

Question 3

Historically $15\%$ of a company's parcels arrive late. After a change of courier, $1$ of $30$ parcels arrives late. Test at the $5\%$ level whether the proportion has decreased.

▶ Show solution

Let $p$ = P(a parcel is late). $H_0: p = 0.15$, $H_1: p < 0.15$. Under $H_0$, $X \sim \mathrm{B}(30,\,0.15)$.

$\mathrm{P}(X \leqslant 1) = 0.0480$.

$0.0480 < 0.05$: reject $H_0$. There is sufficient evidence at the $5\%$ level to suggest that the proportion of late parcels has decreased.

Question 4

Find the critical region for a test of $H_0: p = 0.2$ against $H_1: p > 0.2$ using a sample of $25$, at the $5\%$ level. State the actual significance level.

▶ Show solution

Under $H_0$, $X \sim \mathrm{B}(25,\,0.2)$.

$\mathrm{P}(X \geqslant 8) = 1 - \mathrm{P}(X \leqslant 7) = 0.1091$ ✗

$\mathrm{P}(X \geqslant 9) = 1 - \mathrm{P}(X \leqslant 8) = 0.0468$ ✓

Critical region $X \geqslant 9$; actual significance level $4.68\%$.

Question 5

Find the critical region for a test of $H_0: p = 0.6$ against $H_1: p < 0.6$ using a sample of $20$, at the $5\%$ level. State the actual significance level.

▶ Show solution

Under $H_0$, $X \sim \mathrm{B}(20,\,0.6)$.

$\mathrm{P}(X \leqslant 7) = 0.0210$ ✓, $\mathrm{P}(X \leqslant 8) = 0.0565$ ✗

Critical region $X \leqslant 7$; actual significance level $2.10\%$.

Question 6

Find the critical region for a test of $H_0: p = 0.2$ against $H_1: p \neq 0.2$ using a sample of $30$, at the $5\%$ level. State the actual significance level.

▶ Show solution

Under $H_0$, $X \sim \mathrm{B}(30,\,0.2)$. Each tail $\leqslant 0.025$.

Lower: $\mathrm{P}(X \leqslant 1) = 0.0105$ ✓, $\mathrm{P}(X \leqslant 2) = 0.0442$ ✗ $\Rightarrow X \leqslant 1$.

Upper: $\mathrm{P}(X \geqslant 12) = 1 - \mathrm{P}(X \leqslant 11) = 0.0095$ ✓, $\mathrm{P}(X \geqslant 11) = 0.0256$ ✗ $\Rightarrow X \geqslant 12$.

Critical region: $X \leqslant 1$ or $X \geqslant 12$.

Actual significance level $= 0.0105 + 0.0095 = 0.0200 = 2.00\%$.

Question 7

A coin is thrown $20$ times and shows $5$ heads. Test at the $5\%$ level whether the coin is biased.

▶ Show solution

Let $p$ = P(heads). $H_0: p = 0.5$, $H_1: p \neq 0.5$. Under $H_0$, $X \sim \mathrm{B}(20,\,0.5)$; mean $10$.

$5 < 10$, so use the lower tail: $\mathrm{P}(X \leqslant 5) = 0.0207$.

$0.0207 < 0.025$: reject $H_0$. There is sufficient evidence at the $5\%$ level to suggest that the coin is biased.

Question 8

Find the critical region for a test of $H_0: p = 0.3$ against $H_1: p \neq 0.3$ with $n = 50$, at the $5\%$ level, and the actual significance level.

▶ Show solution

Under $H_0$, $X \sim \mathrm{B}(50,\,0.3)$; mean $15$. Each tail $\leqslant 0.025$.

Lower: $\mathrm{P}(X \leqslant 8) = 0.0183$ ✓, $\mathrm{P}(X \leqslant 9) = 0.0402$ ✗ $\Rightarrow X \leqslant 8$.

Upper: $\mathrm{P}(X \geqslant 23) = 1 - \mathrm{P}(X \leqslant 22) = 0.0123$ ✓, $\mathrm{P}(X \geqslant 22) = 1 - \mathrm{P}(X \leqslant 21) = 0.0251$ ✗ $\Rightarrow X \geqslant 23$.

Note that $0.0251$ is only just over $0.025$ — but it is over, so $22$ is not in the critical region.

Critical region: $X \leqslant 8$ or $X \geqslant 23$. Actual significance level $= 0.01825 + 0.01228 = 0.0305 = 3.05\%$ (adding the unrounded tail probabilities).

Question 9

A politician claims that $40\%$ of voters support a policy. A journalist thinks support is lower and asks $25$ randomly chosen voters. (a) Find the critical region for a test at the $1\%$ level. (b) $5$ voters support the policy. Complete the test.

▶ Show solution

(a) Let $p$ = P(a voter supports the policy). $H_0: p = 0.4$, $H_1: p < 0.4$. Under $H_0$, $X \sim \mathrm{B}(25,\,0.4)$.

$\mathrm{P}(X \leqslant 4) = 0.0095$ ✓, $\mathrm{P}(X \leqslant 5) = 0.0294$ ✗

Critical region $X \leqslant 4$.

(b) $5$ is not in the critical region, so do not reject $H_0$. There is insufficient evidence at the $1\%$ level to suggest that support for the policy is lower than $40\%$.

Question 10

A seed packet states that $85\%$ of the seeds will germinate. Asha thinks that the germination rate of seeds stored in her shed is different. She plants $20$ of these seeds.

(a) State two assumptions needed for a binomial model of the number that germinate. (b) Write down hypotheses and find the critical region at the $10\%$ level. (c) State the actual significance level. (d) $14$ seeds germinate. Complete the test.

▶ Show solution

(a) Seeds germinate independently of one another; each seed has the same probability of germinating. (A fixed number, $20$, is planted.)

(b) Let $p$ = P(a seed germinates). $H_0: p = 0.85$, $H_1: p \neq 0.85$. Under $H_0$, $X \sim \mathrm{B}(20,\,0.85)$; mean $17$. Each tail $\leqslant 0.05$.

Lower: $\mathrm{P}(X \leqslant 13) = 0.0219$ ✓, $\mathrm{P}(X \leqslant 14) = 0.0673$ ✗ $\Rightarrow X \leqslant 13$.

Upper: $\mathrm{P}(X \geqslant 20) = 0.85^{20} = 0.0388$ ✓, $\mathrm{P}(X \geqslant 19) = 0.1756$ ✗ $\Rightarrow X = 20$.

Critical region: $X \leqslant 13$ or $X = 20$.

(c) Actual significance level $= 0.0219 + 0.0388 = 0.0607 = 6.07\%$.

(d) $14$ is not in the critical region, so do not reject $H_0$. There is insufficient evidence at the $10\%$ level to suggest that the germination rate of Asha's seeds is different from $85\%$.

Binomial Hypothesis Tests · A Level Maths Year 1 · Created with MathJax