difficulty 3/5
Huber loss
A loss that behaves like MSE for small errors and like MAE for large ones — smooth enough to optimise easily, robust enough that outliers stop dragging the fit.
Intuition
MSE and MAE each solve one of the other's problems and create their own. MSE is smooth and easy to optimise, but a single wild outlier can drag the whole fit toward it [2, §3.1.1.2]. MAE ignores outliers gracefully, but its kink at zero makes gradient-based optimisation awkward.
Huber loss takes the useful half of each. Close to zero it is quadratic, like MSE — smooth, with a well-behaved gradient. Past a threshold it goes linear, like MAE — so a point ten times further away costs ten times more, not a hundred times [1, description].
Formal definition
How it works
Where it's used
Why & when
Worked example
Pitfalls
Check yourself
3 questions
The questions on this page come with access. They are graded on the server, so the answers stay there until you have given yours.
On the cheat sheet: Regression metrics and losses →
Sources
- scikit-learn — sklearn.linear_model.HuberRegressordocs · retrieved 2026-08-30https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.HuberRegressor.html
- Dive into Deep Learning — 3.1 Linear Regressionbookhttps://d2l.ai/chapter_linear-regression/linear-regression.html
- e-Handbook of Statistical Methods — 1.3.5.17 Detection of Outliersstandard · retrieved 2026-10-04https://www.itl.nist.gov/div898/handbook/eda/section3/eda35h.htm
- Think Stats — Ch. 2, Distributions, 2nd ed.book · retrieved 2026-10-06https://greenteapress.com/thinkstats2/html/thinkstats2003.html