difficulty 3/5
Object detection
Finding every object in an image and drawing a box around each. The output is a variable-length set, which is what makes it harder than classification and why the machinery looks the way it does.
Intuition
Classification returns one label for a whole image. Detection returns a variable-length set of labelled boxes, and almost every awkward thing about the task follows from that one difference. A network emits a fixed-size tensor; a scene contains an unknown number of objects. Something has to bridge that.
The bridge every modern detector uses is the same: propose a large, fixed set of candidate regions, classify each one, adjust its edges, and then delete the duplicates. d2l states the pattern plainly — detection algorithms "usually sample a large number of regions in the input image, determine whether these regions contain objects of interest, and adjust the boundaries of the regions so as to predict the ground-truth bounding boxes of the objects more accurately" [1, §14.4].
Two consequences run through the rest of this page. The candidate set is fixed before training, so it decides what the detector is capable of finding. And the candidate set is enormous while the objects are few, so the training signal is overwhelmingly background.
Formal definition
How it works
Where it's used
Why & when
Worked example
Pitfalls
Check yourself
3 questions
The questions on this page come with access. They are graded on the server, so the answers stay there until you have given yours.
On the cheat sheet: Vision tasks →
Sources
- Dive into Deep Learning — Computer Visionbook · retrieved 2026-09-14https://d2l.ai/chapter_computer-vision/anchor.html
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks (2016)paperhttps://ar5iv.labs.arxiv.org/html/1506.01497
- You Only Look Once: Unified, Real-Time Object Detection (2016)paperhttps://ar5iv.labs.arxiv.org/html/1506.02640
- Focal Loss for Dense Object Detection (2018)paperhttps://ar5iv.labs.arxiv.org/html/1708.02002
- YOLO9000: Better, Faster, Stronger (2017)paperhttps://arxiv.org/abs/1612.08242