difficulty 2/5
Feature engineering
Deciding what the columns are — rescaling, encoding, combining and deriving inputs so that the structure a model can represent is the structure that is actually there.
Intuition
A model can only find structure it is able to express. A linear model asks one question of each column — does the target go up with this? — and if the honest answer is "it goes up, then down, and wraps around at midnight", the column cannot carry it no matter how the coefficients are set.
Feature engineering is the work of changing the question. Not a better optimiser, not more layers: a different column, in which the answer the model can give happens to be the right one. Hour 23 and hour 0 are one hour apart in the world and twenty-three apart in the integer — so you supply a column in which they are adjacent, and the model stops having to fight the encoding.
Its status is easy to underrate because it sits before the modelling and looks like plumbing. Google's practical guidance devotes an entire phase to it, after the first pipeline is running and before anything sophisticated is attempted [2, ML Phase II: Feature Engineering].
Formal definition
How it works
Where it's used
Why & when
Worked example
Pitfalls
Check yourself
3 questions
The questions on this page come with access. They are graded on the server, so the answers stay there until you have given yours.
On the cheat sheet: Missing data and outliers →
Sources
- Dive into Deep Learningbook · retrieved 2026-09-05https://d2l.ai/
- Rules of Machine Learning: Best Practices for ML Engineering (Google)docs · retrieved 2026-09-05https://developers.google.com/machine-learning/guides/rules-of-ml
- Hidden Technical Debt in Machine Learning Systems (2015)paperhttps://proceedings.neurips.cc/paper_files/paper/2015/hash/86df7dcfd896fcaf2674f757a2463eba-Abstract.html
- scikit-learn User Guide — Preprocessing datadocs · retrieved 2026-09-05https://scikit-learn.org/stable/modules/preprocessing.html