difficulty 4/5
Diffusion model
Destroy an image with noise in a thousand fixed steps, then train a network to undo one step at a time. Sampling runs the chain backwards from pure noise.
Intuition
A GAN learns to produce an image in one jump, supervised by an opponent. A diffusion model refuses the jump. It defines a process that destroys an image gradually, and learns only to reverse one small step of it.
The destruction is not learned. "What distinguishes diffusion models from other types of latent variable models is that the approximate posterior q(x₁:T | x₀), called the forward process or diffusion process, is fixed to a Markov chain that gradually adds Gaussian noise to the data according to a variance schedule β₁, …, βT" [1, §2]. Fixed — no parameters, nothing to train, and therefore nothing to destabilise.
What is learned is the reverse: a chain of "learned Gaussian transitions starting at p(xT) = N(0, I)" [1, §2]. Sampling starts from noise and walks back.
Undoing a little noise is an easy, well-posed, supervised problem. Undoing a thousand little bits of noise in sequence turns out to be image generation.
Formal definition
How it works
Where it's used
Why & when
Worked example
Pitfalls
Check yourself
3 questions
The questions on this page come with access. They are graded on the server, so the answers stay there until you have given yours.
On the cheat sheet: Vision tasks →
Sources
- Denoising Diffusion Probabilistic Models (2020)paperhttps://ar5iv.labs.arxiv.org/html/2006.11239
- Generative Adversarial Nets (2014)paperhttps://ar5iv.labs.arxiv.org/html/1406.2661
- U-Net: Convolutional Networks for Biomedical Image Segmentation (2015)paperhttps://ar5iv.labs.arxiv.org/html/1505.04597
- High-Resolution Image Synthesis with Latent Diffusion Models (2022)paperhttps://arxiv.org/abs/2112.10752