The chi-square statistic:
comparing what you saw with what you expected

What the statistic compares

The chi-square statistic, written χ², measures the gap between counts you observed and counts you would expect if a particular assumption were true. If the assumption is that a coin lands heads and tails equally, then 100 flips should give 50 and 50. Getting 30 heads and 70 tails is a gap, and χ² gives that gap a single number.

It works with counts of categories, not with averages. Each category contributes its own squared gap, scaled by what was expected there, and the contributions are added. Bigger gaps and smaller expected counts both push the statistic up.

The worked sum for the 30 and 70 case

Squaring does two jobs. It removes the sign, so being 20 under in one category does not cancel being 20 over in another, and it makes large gaps count disproportionately. Dividing by E stops a gap of 20 on an expectation of 50 looking the same as 20 on an expectation of 5000.

Reading the number: degrees of freedom and critical values

A χ² of 16 means little until you compare it with the value that chance alone would exceed only rarely. That comparison depends on the degrees of freedom (df), which for a goodness-of-fit test is the number of categories minus one. Here it is 2 − 1 = 1.

With 1 df, 16 is far beyond 3.841 and also beyond 6.635. The p-value, the chance of a χ² this large or larger if the coin were fair, is about 0.00006, so the 30/70 split is strong evidence against a fair coin. A different, smaller χ², such as 2, would sit below 3.841 and be consistent with chance variation.

A two-way table: expected counts from the margins

For a table that cross-classifies two variables, the expected count in each cell comes from the row and column totals: row total × column total ÷ grand total. Suppose a store shows two versions of a page, each to 100 visitors, and 30 visitors buy after version A and 45 after version B.

A multi-category case: is a die fair?

Roll a die 60 times and record how often each face appears. A fair die gives an expected count of 10 for each of the six faces. Suppose you observe 8, 12, 9, 11, 10 and 10.

• (8 − 10)² ÷ 10 = 0.4
• (12 − 10)² ÷ 10 = 0.4
• (9 − 10)² ÷ 10 = 0.1
• (11 − 10)² ÷ 10 = 0.1
• 10 and 10 contribute 0 each
• Total χ² = 1.0 with 5 df

The 5% critical value for 5 df is 11.07, so a χ² of 1.0 is entirely consistent with a fair die. Note that very small values are also informative. An unnaturally small χ² can suggest figures that were smoothed or invented, since real randomness always produces some spread.

Statistical significance is not the size of the effect

A χ² value, even one that passes the critical value, says only that the gap is unlikely to be chance. It does not tell you whether the gap is large enough to matter. Effect-size measures put the statistic back on a scale of zero to one. Cramér's V is √(χ² ÷ (n × k)), where n is the number of observations and k is the smaller of rows − 1 and columns − 1.

• Two-version page test: V = √(4.8 ÷ (200 × 1)) = 0.155, a modest effect.
• Coin example: V = √(16 ÷ (100 × 1)) = 0.40, a large effect.
• A very large survey might give a significant χ² with V of 0.03, a difference that is real but of little practical weight.

There is also a link to the z-test for a proportion. With two categories, χ² equals the square of the z-score. For the coin, z = (30 − 50) ÷ √(100 × 0.5 × 0.5) = −4, and (−4)² = 16, which is exactly the χ² you computed. The two tests give identical p-values in that case.

When the approximation is unreliable

• Small expected counts: a common rule is that every expected count should be at least 5, or at least 80% of cells should be 5 or more.
• Dependent observations: the same person counted twice breaks the assumption of independence.
• Not counts: percentages and averages must be turned back into counts first.
• Tiny 2 × 2 tables: Fisher's exact test is preferred when expected counts are small.
• Huge samples: with enough data, trivial differences become significant, so look at the size of the effect too.

One more habit helps: write the hypothesis in words before computing anything. 'The die is fair' and 'version B converts no differently from version A' are testable statements, and they tell you which expected counts to build. Decide in advance which significance level you will use, commonly 5%, rather than choosing it after you see the p-value, since picking the threshold afterwards quietly turns a test into a justification.

A chi-square statistic calculator performs the arithmetic and looks up the p-value, but it cannot check these conditions for you. State the assumption being tested, confirm the counts are independent, and report both the statistic and the degrees of freedom. For anything with real consequences, have a statistician look over the design.