difficulty 3/5
PCA
A rotation of the axes onto the directions of greatest spread. It makes the data uncorrelated and rank-orders the new axes, so you can drop the last ones and know exactly what that costs.
Intuition
Measure the same thing three ways and you have three columns carrying roughly one column's worth of information. The tutorial this page follows calls that redundancy, and puts the observation plainly: when two measurements are highly correlated "one can calculate r₁ from r₂ (or vice versa) using the best-fit line. Recording solely one response would express the data more concisely … Indeed, this is the central idea behind dimensional reduction" [1, §IV.2].
The second observation is about which directions matter. In a cloud stretched along a diagonal, "the direction with the largest variance is not x̂ nor ŷ, but the direction along the long axis of the cloud" [1, §IV.1]. The axes you happened to measure in are arbitrary; the axes the data is actually spread along are not.
PCA does one thing about both: it rotates. It finds a new set of perpendicular axes, ordered so the first has the most spread, and in that basis the columns are uncorrelated. Nothing is thrown away by the rotation itself — throwing away comes after, and the ordering tells you exactly what it costs.
Formal definition
How it works
Where it's used
Why & when
Worked example
Pitfalls
Check yourself
3 questions
The questions on this page come with access. They are graded on the server, so the answers stay there until you have given yours.
Sources
- A Tutorial on Principal Component Analysis (2014)paperhttps://ar5iv.labs.arxiv.org/html/1404.1100
- scikit-learn User Guide — Clusteringdocs · retrieved 2026-09-06https://scikit-learn.org/stable/modules/clustering.html