Margin of error:
how much a survey percentage can wobble

What a margin of error does and does not say

A survey asks a sample, not everyone, so its percentage is an estimate with some wobble. The margin of error is the half-width of the interval around the sample result that is expected to contain the true population value, at a stated confidence level, typically 95 percent.

If a poll reports 52 percent support with a margin of error of ±3.1 points, the plausible range for the whole population is roughly 48.9 to 55.1 percent. That is not a guarantee, because at 95 percent confidence about one sample in twenty would miss.

Crucially, the margin covers only random sampling variation. It says nothing about a badly worded question, people who refuse to answer, or a sample that is not random. A poll can have a tiny margin of error and still be wrong.

The formula and its ingredients

The term p(1 − p) is largest at p = 0.5, which is why pollsters quote the worst-case margin using 50 percent when the true value is unknown. It becomes smaller as p moves toward 0 or 1, because a result near 5 percent or 95 percent has less room to vary.

The sample size sits under a square root, which creates the central trade-off in survey design: a larger sample helps, but with diminishing returns.

Worked example: n = 1,000 at 95 percent

A market survey asks 1,000 randomly chosen adults and finds 50 percent in favour. Use p = 0.5 and z = 1.96.

For comparison, the same calculation with n = 400 gives 1.96 × √(0.25 ÷ 400) = 0.049, or ±4.9 points. With p = 0.2 and n = 1,000 the margin shrinks to about ±2.5 points.

How the sample size changes the margin

To go from ±3.1 to ±1.5 points you need four times the respondents, from 1,000 to 4,000. Doubling the work from 1,000 to 2,000 only brings the margin down to ±2.2.

Inverting the formula gives the sample size you need for a target margin E: n = z² × p(1 − p) ÷ E². For ±3 points at 95 percent, n = 1.96² × 0.25 ÷ 0.03² = 1,067.1, so round up to 1,068. For ±5 points it is 385, and for ±2 points 2,401.

Confidence level and small populations

Changing the confidence level changes z. With n = 1,000 and p = 0.5, a 90 percent level (z = 1.645) gives ±2.6 points, 95 percent gives ±3.1, and 99 percent (z = 2.576) gives ±4.1. A wider interval is the price of greater certainty.

If the sample is a large part of a small population, the margin is smaller than the standard formula suggests. The finite population correction multiplies it by √[(N − n) ÷ (N − 1)]. For N = 5,000 and n = 1,000 the factor is 0.894, and the margin falls from ±3.1 to about ±2.8 points.

Margin of error for an average

Percentages are not the only quantity with a margin. For the mean of a measurement such as time, weight or score, the margin of error is the critical value times the standard error of the mean, which is the sample standard deviation divided by the square root of n.

A survey of 64 customers finds an average delivery time with a standard deviation of 12 minutes. The margin of error is 1.96 × 12 ÷ √64 = 1.96 × 12 ÷ 8 = 2.94 minutes. If the mean is 45 minutes, the true average is likely between about 42.1 and 47.9 minutes.

With small samples the normal z value is too optimistic, and the t distribution is used instead. For n = 16 and the same standard deviation, the 95 percent t value with 15 degrees of freedom is 2.131, and the margin becomes 2.131 × 12 ÷ 4 = 6.4 minutes, more than twice as wide, even though the sample is only a quarter the size.

Reading results sensibly

• A lead of 52 to 48 with ±3.1 on each figure is within the noise; the gap needs to exceed roughly twice the margin for the lead to be clear.
• Subgroups have fewer respondents and so larger margins; the margin for men only is bigger than the margin for everyone.
• Compare to earlier polls of the same design, since changes within the margin are often just sampling variation.
• Do not forget the non-sampling errors: coverage, wording, non-response and weighting.

A calculator is useful for working backwards. Enter a target margin and it returns the sample you need, which is how survey budgets are usually set.