difficulty 2/5
KNN
Classify a new point by looking at the k closest training examples and taking their majority vote — a model that stores the data instead of learning from it.
Intuition
Ask the neighbours. To classify something new, find the k training examples closest to it and let them vote.
There is no fitting step worth the name — the model is the training set: "there is not even really a training stage or learning process" [1, §5.7.3]. That makes KNN the clearest illustration of a non-parametric method: it assumes nothing about the shape of the boundary, and so it is "often successful in classification situations where the decision boundary is very irregular" [3, §1.6].
Formal definition
How it works
Where it's used
Why & when
Worked example
Pitfalls
Check yourself
3 questions
The questions on this page come with access. They are graded on the server, so the answers stay there until you have given yours.
Sources
- Deep Learning (2016)book · retrieved 2026-10-07https://www.deeplearningbook.org/contents/ml.html
- scikit-learn — KNeighborsClassifierdocs · retrieved 2026-08-31https://scikit-learn.org/stable/modules/generated/sklearn.neighbors.KNeighborsClassifier.html
- scikit-learn user guide — Nearest Neighborsdocs · retrieved 2026-10-07https://scikit-learn.org/stable/modules/neighbors.html
- scikit-learn — KNNImputerdocs · retrieved 2026-10-07https://scikit-learn.org/stable/modules/generated/sklearn.impute.KNNImputer.html