difficulty 3/5
Manifold learning
Non-linear projections that keep neighbours together at the cost of everything else. They make wonderful pictures, and almost nothing you can measure off those pictures is real.
Intuition
PCA rotates. That is its strength — the answer is unique, the axes mean something — and its ceiling: a rotation cannot flatten a circle. When the low-dimensional structure in the data is curved, a linear method has nothing to offer.
Manifold methods drop the requirement that the map be linear, and replace it with a different one: points that are near each other in the original space should be near each other in the picture. Everything else is negotiable, and in practice everything else is negotiated away [1, §6].
This is why t-SNE and UMAP plots are simultaneously the most persuasive and the most misleading figures in applied machine learning. They really do reveal cluster structure that nothing else finds. They also produce plots where the size of a blob, the gap between two blobs, and the shape of anything are all artefacts — and the plot gives no sign of which is which.
Formal definition
How it works
Where it's used
Why & when
Worked example
Pitfalls
Check yourself
3 questions
The questions on this page come with access. They are graded on the server, so the answers stay there until you have given yours.
Sources
- UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction (2020)paperhttps://ar5iv.labs.arxiv.org/html/1802.03426
- How to Use t-SNE Effectively (2016)paperhttps://distill.pub/2016/misread-tsne/