Classification accuracy:
formula and worked example

Counting right answers

Classification accuracy is the fraction of all predictions that match the true label. It treats every case the same: a correct prediction scores one, a wrong one scores zero, and the result is the average. That simplicity is why it is usually the first number reported for a classifier.

It works well as a quick health check when the classes are roughly balanced and when the cost of different mistakes is similar. Sorting photos into cat or dog on a balanced test set is a fair example.

It becomes far less informative when one outcome is much more common than the others, or when a missed case costs far more than a false alarm. In those settings the single number can look impressive while the model is practically useless.

Formula and inputs

• Correct: the number of cases where the predicted label equals the true label, summed over every class.
• Total: the number of cases evaluated, including the ones the model got wrong.

In a binary confusion matrix, correct is true positives plus true negatives, and total adds the false positives and false negatives. Leaving out cases the model abstained on changes the denominator, so be explicit about how those are treated.

Accuracy also assumes each case counts equally. If the test set repeats some customers or documents many times, the effective weights differ and the headline figure can be skewed. Deduplicate, or at least state that the set was weighted by occurrence.

Worked example: 870 of 1,000

A model labels 1,000 test emails as spam or not spam and matches the true label on 870 of them.

1. 1Count correct predictions: 870.
2. 2Count all predictions: 1,000.
3. 3Divide: 870 / 1,000 = 0.87.
4. 4Multiply by 100 to express as a percentage: 87%.

Is 87% good?

The answer depends on the baseline. If the test set is split evenly, 87% is comfortably better than the 50% a coin toss gives. If 90% of the emails are not spam, a filter that labels everything as not spam scores 90% and the model is worse than doing nothing.

So the practical routine is to compute the majority-class accuracy first and treat the model's accuracy as meaningful only to the extent it exceeds it. The gap, or the proportion of the error removed, tells you more than the raw percentage.

Beyond the baseline, look at the confusion matrix. Two models with the same 87% can make very different kinds of mistakes, and which kind matters depends on the application.

Uncertainty deserves a mention too. An accuracy of 87% on 1,000 cases carries a margin of roughly two percentage points either side at the usual 95% confidence level, so a rival model at 88% is not clearly better. Differences of one or two points on a test set of this size are often just noise, and paired comparisons on the same cases give a firmer answer.

Gaining correct predictions

Going from 870 to 957 correct predictions lifts accuracy from 87% to 95.7%, an improvement of 8.7 points. Notice that the 10% rise in correct answers cuts the error count from 130 to 43, a two-thirds reduction. Accuracy moves slowly near the top of the scale in percentage terms but the error rate falls fast, which is why teams often report the error rate when comparing strong models.

Common ways accuracy misleads

• Ignoring class imbalance, so a trivial model looks excellent.
• Measuring on data the model was trained on, which overstates performance on new cases.
• Treating all errors as equal when a missed fraud or missed illness costs far more than a false alarm.
• Comparing accuracy across datasets whose class mix differs, as the same model will score differently on each.
• Rounding or reporting a figure from a small test set without noting the uncertainty. 87% from 50 cases is much less settled than from 5,000.

A last point is label quality. If the true labels themselves are wrong in a few per cent of cases, accuracy has a ceiling below 100% and chasing it can mean fitting the errors in the labels rather than the underlying pattern.

A realistic check

A hospital tests a model on 2,000 scans where 100 show the target finding. The model labels all 2,000 as normal. Accuracy is 1,900 / 2,000 = 95%, which sounds strong, yet sensitivity is zero. A second model gets 1,850 right but finds 80 of the 100 true cases. Its accuracy is lower, at 92.5%, but it is plainly the more useful one.

The same discipline applies when choosing between candidate models. Fix the test set before looking at any results, compare every candidate on the same cases, and decide in advance which error type matters most. Picking the highest accuracy from a dozen tries on one test set, then reporting that figure, is a quiet way of fooling yourself, because the winner is partly selected for luck.