Which error, in which units?
Seven regression metrics and losses side by side, the pairs people confuse, and one run in which the same errors read six different ways.
prints on 2 A4 pagesfrom 7 pages and 1 cataloguefree to read · PDF with a free accountUpdated 21 Oct 2026
The PDF and the Print button ask you to sign in; a free account is enough. Reading and Test yourself are open to everyone.
Side by side
The first columns are the Regression metrics catalogue’s; the rest come from each page.
A formula is printed only when it is shorter than the words; the others are said in words, and each page has its formula.
Easy to confuse
The “what is the difference between…” questions, in two lines each.
5.20 vs 2.72 Case: one-outlier
RMSE squares before it averages, so a few big misses pull it up; MAE counts every unit of error the same.
Report both. A large gap between them means a few big misses dominate.
29.98 vs 0.93 Case: x10-units
RMSE is an error in the target's units; R² is the share of the variance explained, with no units at all.
R² says whether the model beats the mean; RMSE says how far off it is. Report both.
MAPE divides each error by the true value; MAE does not, so near zero MAPE blows up on errors MAE barely notices.
MAPE only for targets well away from zero; MAE for anything that can reach or cross it.
MSE vs Huber loss
Both square the small errors; past δ Huber grows linearly, so one wild point cannot dominate it.
Huber when outliers stay in the data; MSE when the data is clean.
Both measure relative error. MSLE works on logs with 1 added, so a zero target does not break it.
MSLE when the target spans orders of magnitude; MAPE when the audience thinks in percent.
One real run
The same 200 predictions, while one thing changes at a time.
| Case | MSE | RMSE | MAE | R² | MAPE | Huber |
|---|---|---|---|---|---|---|
| clean | 8.99 | 3.00 | 2.42 | 0.93 | 0.05 | 4.40 |
| one-outlier | 27.07 | 5.20 | 2.72 | 0.78 | 0.06 | 5.84 |
| near-zero | 8.99 | 3.00 | 2.42 | 0.93 | 0.22 | 4.40 |
| x10-units | 898.78 | 29.98 | 24.22 | 0.93 | 0.05 | 109.03 |
“One bad prediction triples MSE and barely moves MAE.”
“Move the target near zero and only MAPE moves.”
“Change the units and only R² and MAPE stay put.”
The run and these lines are the worked example on MSE, as it printed: 200 targets, the same noise in every row; Huber delta = 5
Which one?
Is a big miss worse than its size?
Who reads the number?
Is the error relative to the size of the target?
Outliers you cannot remove, and gradient training?
- Yes Huber loss
Each answer is from its page’s “Why & when”.
All 25 traps on these pages
Each is the opening line of a pitfall; its page says why.
MSE
- MSE is not comparable across datasets.
- Training MSE is not a performance estimate.
- The squaring is a decision, not a technicality.
RMSE
- RMSE is still not comparable across datasets.
- It is not the average error.
- Taking the root does not add robustness.
MAE
- MAE is not comparable across datasets.
- Low MAE can hide systematic bias.
- It resists outliers rather than handling them.
R²
- High R² is not evidence the model is right.
- Training R² only ever goes up.
- R² is not the square of the correlation in general.
- Negative values are normal on held-out data.
MAPE
- A single zero ruins it.
- It is asymmetric and biases models downward.
- scikit-learn returns a fraction, not a percent.
- It is undefined for targets that cross zero.
MSLE
- Negative values do not always raise an error.
- It is not symmetric, and that is deliberate.
- The number is not interpretable on its own.
- It quietly changes what the model optimises for.
Huber loss
- δ is scale-dependent.
- Robust is not immune.
- Do not use it to avoid looking at your data.
- The reported number is not comparable to MSE or MAE.