difficulty 2/5
Sigmoid
An S-shaped activation that squashes any real number into (0, 1) — the historical default for neural networks, now largely displaced from hidden layers because it strangles the gradient.
Intuition
The perceptron's step function is brutal: everything below the threshold is 0, everything above is 1, and the jump between them is instantaneous. That makes it undifferentiable exactly where it matters, so gradient-based training is impossible.
Sigmoid is the smooth version of that step. Same shape, same limits, but the transition is gradual — which means it has a derivative everywhere, and networks can be trained.
Formal definition
How it works
Where it's used
Why & when
Worked example
Pitfalls
Check yourself
3 questions
The questions on this page come with access. They are graded on the server, so the answers stay there until you have given yours.
On the cheat sheet: Activation functions →
Sources
- Dive into Deep Learningbook · retrieved 2026-08-31https://d2l.ai/
- Deep Learning (2016)bookhttps://www.deeplearningbook.org/
- Efficient BackProp (1998)paperhttps://doi.org/10.1007/3-540-49430-8_2
- On Calibration of Modern Neural Networks (2017)paperhttps://arxiv.org/abs/1706.04599