Hypothesis Testing Overview – A Level Maths Year 1

1 The Big Idea

Hypothesis testing (DfE content area O · Statistical hypothesis testing) is how a statistician decides whether a piece of sample evidence is strong enough to doubt a claim about a population. At AS level the claim is always about a probability $p$, and the evidence is a count that follows a binomial distribution.

Start by assuming the claim is true. Work out how likely the observed result (or something even more extreme) would be under that assumption. If it would be very unlikely, you have evidence against the claim. If it would not be particularly unusual, you have no reason to doubt it.

The logic in one line

If $\mathrm{P}(\text{result this extreme} \mid H_0 \text{ true}) \leqslant$ significance level, reject $H_0$

Worked Example — Is the coin biased?

Sam suspects a coin is biased towards heads. She throws it $20$ times and gets $15$ heads. Test at the $5\%$ significance level.

①Let $p$ = probability that the coin shows heads. $H_0: p = 0.5$, $H_1: p > 0.5$

②Let $X$ = number of heads in $20$ throws. Under $H_0$, $X \sim \mathrm{B}(20,\,0.5)$.

③$\mathrm{P}(X \geqslant 15) = 1 - \mathrm{P}(X \leqslant 14) = 0.0207$ (to 4 d.p.)

④$0.0207 < 0.05$, so the result is significant: reject $H_0$.

⑤There is sufficient evidence at the $5\%$ level to suggest that the coin is biased towards heads.

Notice the word suggest. Fifteen heads does not prove bias — a fair coin does this about $2\%$ of the time. The test only tells you the evidence is strong enough to doubt fairness.

Calculators are allowed, and you are expected to use the binomial cumulative distribution function on yours. Always write down the probability statement (for example $\mathrm{P}(X \geqslant 15)$) and the distribution before you type anything in — those lines earn the method marks.

2 The Language You Need

The vocabulary of hypothesis testing is examinable in its own right. Questions regularly ask you to "state", "define" or "explain" these terms.

Term	Meaning
Null hypothesis $H_0$	The claim you assume to be true; always gives an exact value, e.g. $p = 0.3$.
Alternative hypothesis $H_1$	What you suspect instead: $p > 0.3$, $p < 0.3$ or $p \neq 0.3$.
Significance level	The threshold probability for "very unlikely", e.g. $5\%$.
Test statistic	The quantity calculated from the sample, e.g. $X$ = number of successes.
One-tailed test	$H_1$ looks in one direction only ($>$ or $<$).
Two-tailed test	$H_1$ is $p \neq \ldots$; extreme results in either direction count.
Critical region	The set of values of the test statistic that lead to rejecting $H_0$.
Critical value	The first value inside the critical region.
Acceptance region	The values that do not lead to rejecting $H_0$.
p-value	The probability, under $H_0$, of a result at least as extreme as the one observed.
Actual significance level	The true probability of the critical region; the probability of wrongly rejecting $H_0$.

Hypotheses are about the population, not the sample. Write $H_0: p = 0.5$, where $p$ is the probability for the whole population. Never write $H_0: X = 10$ — $X$ is your sample result.

3 The Core Procedure

Every binomial hypothesis test follows the same six steps. Learn them as a checklist.

Define the parameter in words: "Let $p$ be the probability that …".
State the hypotheses $H_0$ and $H_1$ in terms of $p$.
State the distribution of the test statistic under $H_0$: $X \sim \mathrm{B}(n,\,p)$.
Calculate the p-value, or find the critical region.
Compare with the significance level (halved in each tail for a two-tailed test).
Conclude in context, using non-assertive language.

Two ways to reach the decision

p-value: reject $H_0$ if $\mathrm{P}(\text{observed or more extreme}) \leqslant \alpha$
Critical region: reject $H_0$ if the observed value lies in the critical region

The calculator identity you need for upper tails

$\mathrm{P}(X \geqslant k) = 1 - \mathrm{P}(X \leqslant k - 1)$

Worked Example — A lower-tail test

A drug company claims that $60\%$ of patients recover within a week when given its treatment. A doctor believes the true proportion is lower. Of $25$ patients, $10$ recover within a week. Test the doctor's belief at the $5\%$ level.

①Let $p$ = probability that a patient recovers within a week.

②$H_0: p = 0.6$, $H_1: p < 0.6$

③Under $H_0$, $X \sim \mathrm{B}(25,\,0.6)$.

④$\mathrm{P}(X \leqslant 10) = 0.0344$

⑤$0.0344 < 0.05$, so reject $H_0$.

⑥There is sufficient evidence at the $5\%$ level to suggest that fewer than $60\%$ of patients recover within a week.

4 The Hypothesis Testing Subtopics

DfE content area O at AS level, and the page that covers each part. You will need the binomial distribution from The Binomial Distribution throughout.

OThe Language of Hypothesis TestingNull and alternative hypotheses, significance level, critical regions, p-values and how to write a conclusion.
OBinomial Hypothesis TestsOne- and two-tailed tests for a proportion, by p-value and by critical region.

5 Common Mistakes to Avoid

Using $\mathrm{P}(X = x)$ instead of a tail probability. The p-value is the probability of the observed value or anything more extreme. For $15$ heads you need $\mathrm{P}(X \geqslant 15)$, not $\mathrm{P}(X = 15)$.

Getting the upper tail wrong on a calculator. $\mathrm{P}(X \geqslant 15) = 1 - \mathrm{P}(X \leqslant 14)$, not $1 - \mathrm{P}(X \leqslant 15)$.

Forgetting to halve for a two-tailed test. At the $5\%$ level with $H_1: p \neq 0.4$, each tail gets $2.5\%$.

Over-confident conclusions. Never write "this proves the coin is biased" or "we accept $H_0$". Write "there is sufficient evidence to suggest…" or "there is insufficient evidence to suggest…".

Conclusions with no context. "Reject $H_0$" on its own scores only part of the marks. Say what it means for the coin, the drug, the seeds…

6 Quick Reference

Hypotheses

$H_0: p = p_0$; $H_1: p > p_0$, $p < p_0$ or $p \neq p_0$.

Test statistic

$X \sim \mathrm{B}(n,\,p_0)$ under $H_0$.

Upper tail

$\mathrm{P}(X \geqslant k) = 1 - \mathrm{P}(X \leqslant k-1)$.

p-value decision

p-value $\leqslant \alpha$ $\Rightarrow$ reject $H_0$.

Critical region

Largest region with probability $\leqslant \alpha$ (per tail).

Two-tailed

Put $\tfrac{\alpha}{2}$ in each tail.

Actual significance level

$\mathrm{P}(X \in \text{critical region} \mid H_0)$.

Conclusion

"Sufficient / insufficient evidence to suggest…" + context.

Which tail? (two-tailed)

Compare the observed value with the mean $np_0$.

7 Practice Questions

Ten questions drawing on the whole topic. Use the binomial functions on your calculator, and give probabilities to 4 decimal places.

Question 1

A die is thrown repeatedly to test whether it shows a six less often than a fair die. Define a suitable parameter and write down the null and alternative hypotheses.

▶ Show solution

Let $p$ be the probability that the die shows a six on any throw.

$H_0: p = \tfrac{1}{6}$, $H_1: p < \tfrac{1}{6}$

Question 2

For each situation, state whether the test should be one-tailed or two-tailed. (a) A teacher wants to know whether a new revision method improves the pass rate of $70\%$. (b) A quality controller wants to know whether the proportion of faulty items has changed from $4\%$.

▶ Show solution

(a) One-tailed: the teacher is only interested in an increase. $H_1: p > 0.7$.

(b) Two-tailed: "changed" could mean up or down. $H_1: p \neq 0.04$.

Question 3

$X \sim \mathrm{B}(12,\,0.5)$. Find $\mathrm{P}(X \geqslant 10)$.

▶ Show solution

$\mathrm{P}(X \geqslant 10) = 1 - \mathrm{P}(X \leqslant 9)$

$= 1 - 0.9807 = 0.0193$ (to 4 d.p.)

Question 4

Explain why a conclusion should never say "we accept $H_0$" or "this proves $H_1$".

▶ Show solution

A hypothesis test works with probabilities from a single sample. An unusual result can happen by chance even when $H_0$ is true, and an ordinary result can happen even when $H_0$ is false.

So the test can only say whether there is sufficient evidence to doubt $H_0$. It never proves either hypothesis. Failing to reject $H_0$ means only that there is insufficient evidence against it.

Question 5

It is claimed that $20\%$ of customers buy a dessert. A manager thinks the proportion is higher. In a sample of $15$ customers, $6$ buy a dessert. Test the manager's belief at the $5\%$ level.

▶ Show solution

Let $p$ = probability that a customer buys a dessert. $H_0: p = 0.2$, $H_1: p > 0.2$.

Under $H_0$, $X \sim \mathrm{B}(15,\,0.2)$.

$\mathrm{P}(X \geqslant 6) = 1 - \mathrm{P}(X \leqslant 5) = 0.0611$

$0.0611 > 0.05$, so do not reject $H_0$.

There is insufficient evidence at the $5\%$ level to suggest that more than $20\%$ of customers buy a dessert.

Question 6

Historically $15\%$ of a company's parcels arrive late. After a change of courier, $1$ parcel out of a random sample of $30$ arrives late. Test at the $5\%$ level whether the proportion of late parcels has decreased.

▶ Show solution

Let $p$ = probability that a parcel arrives late. $H_0: p = 0.15$, $H_1: p < 0.15$.

Under $H_0$, $X \sim \mathrm{B}(30,\,0.15)$.

$\mathrm{P}(X \leqslant 1) = 0.0480$

$0.0480 < 0.05$, so reject $H_0$.

There is sufficient evidence at the $5\%$ level to suggest that the proportion of late parcels has decreased. (Only just — at the $1\%$ level the conclusion would be different.)

Question 7

$X \sim \mathrm{B}(10,\,0.5)$. Find the critical region for a test of $H_0: p = 0.5$ against $H_1: p > 0.5$ at the $5\%$ level, and state the actual significance level.

▶ Show solution

$\mathrm{P}(X \geqslant 9) = 1 - \mathrm{P}(X \leqslant 8) = 0.0107$ — below $0.05$.

$\mathrm{P}(X \geqslant 8) = 1 - \mathrm{P}(X \leqslant 7) = 0.0547$ — above $0.05$.

Critical region: $X \geqslant 9$. Actual significance level $= 0.0107$, i.e. $1.07\%$.

Question 8

A coin is thrown $20$ times to test $H_0: p = 0.5$ against $H_1: p \neq 0.5$, where $p$ is the probability of heads. It shows $5$ heads. Carry out the test at the $5\%$ level.

▶ Show solution

Under $H_0$, $X \sim \mathrm{B}(20,\,0.5)$, with mean $10$. The observed $5$ is below the mean, so use the lower tail.

$\mathrm{P}(X \leqslant 5) = 0.0207$

Two-tailed, so compare with $0.025$: $0.0207 < 0.025$, reject $H_0$.

There is sufficient evidence at the $5\%$ level to suggest that the coin is biased.

Question 9

A test of $H_0: p = 0.3$ against $H_1: p > 0.3$ uses a sample of $20$ and has critical region $X \geqslant 10$. (a) Find the actual significance level. (b) The observed value is $9$. State the conclusion.

▶ Show solution

(a) Under $H_0$, $X \sim \mathrm{B}(20,\,0.3)$. $\mathrm{P}(X \geqslant 10) = 1 - \mathrm{P}(X \leqslant 9) = 1 - 0.9520 = 0.0480$, i.e. $4.80\%$.

(b) $9$ is not in the critical region, so do not reject $H_0$. There is insufficient evidence to suggest that $p$ is greater than $0.3$.

Question 10

A seed merchant claims that $40\%$ of a type of bulb produce red flowers. A gardener plants $20$ bulbs to test whether the claim is wrong in either direction.

(a) Write down suitable hypotheses. (b) Find the critical region at the $5\%$ level. (c) State the actual significance level. (d) $13$ of the bulbs produce red flowers. Complete the test.

▶ Show solution

(a) Let $p$ = probability that a bulb produces a red flower. $H_0: p = 0.4$, $H_1: p \neq 0.4$.

(b) Under $H_0$, $X \sim \mathrm{B}(20,\,0.4)$. Each tail may have at most $0.025$.

Lower tail: $\mathrm{P}(X \leqslant 3) = 0.0160$ ✓, $\mathrm{P}(X \leqslant 4) = 0.0510$ ✗. So $X \leqslant 3$.

Upper tail: $\mathrm{P}(X \geqslant 13) = 1 - \mathrm{P}(X \leqslant 12) = 0.0210$ ✓, $\mathrm{P}(X \geqslant 12) = 0.0565$ ✗. So $X \geqslant 13$.

Critical region: $X \leqslant 3$ or $X \geqslant 13$.

(c) Actual significance level $= 0.0160 + 0.0210 = 0.0370$, i.e. $3.70\%$.

(d) $13$ lies in the critical region, so reject $H_0$. There is sufficient evidence at the $5\%$ level to suggest that the merchant's claim of $40\%$ is wrong.

Hypothesis Testing Overview · A Level Maths Year 1 · Created with MathJax