difficulty 3/5
Maximum likelihood
Pick the parameters that make the observed data as probable as possible. Nearly every loss function in machine learning is this rule with a minus sign and a logarithm in front of it.
Intuition
You flip a coin thirteen times and get nine heads. What is the coin's bias?
Everyone answers 9/13 without thinking. Maximum likelihood is the machinery that gets to the same answer from first principles, "in a way that will generalize to vastly more complex situations" [1, §22.7.1.1] — and the generalisation is the point, because in the complex situations there is no obvious answer to check against.
The rule itself is one sentence: "the parameters which make the data have the highest probability are the most likely ones" [1, §22.7]. Turn the usual question round. Instead of asking how probable an outcome is given the parameters, fix the outcome you actually saw and ask which parameters would have made it likeliest.
Formal definition
How it works
Where it's used
Why & when
Worked example
Pitfalls
Check yourself
3 questions
The questions on this page come with access. They are graded on the server, so the answers stay there until you have given yours.
Sources
- Dive into Deep Learningbook · retrieved 2026-09-12https://d2l.ai/chapter_appendix-mathematics-for-deep-learning/maximum-likelihood.html
- Think Stats, 2nd ed.book · retrieved 2026-09-12https://greenteapress.com/thinkstats2/html/index.html