Accuracy

Intuition

Count how many predictions were right, divide by how many you made. That is accuracy, and it is the first metric everybody reaches for because it is the one that needs no explanation.

It is also the one most likely to mislead you, for a single reason: it treats every mistake as identical and every class as equally common. Both assumptions fail routinely.

Formal definition

scikit-learn computes the fraction of correctly classified samples [1, definition]. In terms of the confusion matrix, it is the diagonal divided by the whole table.

With normalize=False the same function returns the raw count instead of the fraction [1, normalize].

In multilabel problems it becomes subset accuracy: the predicted label set must match the true label set exactly, which is a far stricter test than it sounds [1, definition].

How it works

Accuracy is one number summarising the entire confusion matrix, which means it compresses four counts into one — and compression loses information. Two models with identical accuracy can behave in opposite ways: one refusing to predict the positive class at all, the other flagging everything.

Where it's used

• Balanced benchmarks. Where classes are roughly equal and errors cost the same, accuracy is a fair summary and an easy one to communicate.
• A first sanity check. Comparing accuracy against the majority-class base rate tells you immediately whether a model has learned anything.
• Reporting to non-specialists. It is the only classification metric that needs no preamble.

Why & when

Use it when the classes are balanced and the two kinds of error cost about the same. Under those conditions it is genuinely the right summary.

Reach for something else the moment the data is skewed. In a dataset that is 99% negative, predicting "negative" every time scores 99% accuracy while being completely useless [4, §8.3]. Precision, recall and the confusion matrix are what survive class imbalance; accuracy does not [2, Accuracy].

Worked example

from sklearn.metrics import accuracy_score

y_pred = [0, 2, 1, 3]
y_true = [0, 1, 2, 3]

accuracy_score(y_true, y_pred)
# 0.5
accuracy_score(y_true, y_pred, normalize=False)
# 2.0

Two of four predictions are correct — positions 0 and 3 — so accuracy is 0.5 [1, examples]. Note that the two wrong predictions were a straight swap of classes 1 and 2, which accuracy cannot show you and a confusion matrix would.

Pitfalls

Always compare against the base rate. An accuracy of 95% is excellent if the majority class is 50% of the data and worthless if it is 96%. The number alone is uninterpretable [2, Accuracy].

Accuracy hides which error the model makes. Two models with the same accuracy can fail in opposite ways — one missing the positives, the other raising false alarms — and when one kind of mistake costs more than the other, which is "the case in most real-world applications", that difference is the whole question [2, Accuracy].

It ignores confidence entirely. A prediction made at 51% certainty counts exactly like one made at 99% [3, classification threshold]. If calibrated probabilities matter to your decision, accuracy is not measuring the thing you care about.