difficulty 3/5
Class imbalance
One class is rare, and every default in the toolchain assumes it is not. The accuracy looks excellent, the model predicts the rare class never, and most of the standard fixes only move the threshold.
Intuition
A fraud model reports 99.9% accuracy. It is also, on inspection, predicting "not fraud" for every transaction it has ever seen. Both facts are true at once, and the first one is the reason nobody checked the second.
This is not a subtle failure mode. It is what a loss function does when told that one class is a thousand times more common than the other: getting the common one right is where nearly all the available loss reduction is. The model is behaving correctly and the objective was wrong.
scikit-learn's own documentation treats a suspiciously ordinary score as a symptom worth investigating: "when the accuracy of a classifier is too close to random, it probably means that something went wrong: features are not helpful, a hyperparameter is not correctly tuned, the classifier is suffering from class imbalance" [2, §3.4.8].
Formal definition
How it works
Where it's used
Why & when
Worked example
Pitfalls
Check yourself
3 questions
The questions on this page come with access. They are graded on the server, so the answers stay there until you have given yours.
On the cheat sheet: Classification metrics →
Sources
- A systematic study of the class imbalance problem in convolutional neural networks (2018)paperhttps://ar5iv.labs.arxiv.org/html/1710.05381
- scikit-learn User Guide — Metrics and scoringdocs · retrieved 2026-09-13https://scikit-learn.org/stable/modules/model_evaluation.html