difficulty 2/5
Tanh
A rescaled sigmoid that outputs between −1 and 1 instead of 0 and 1 — zero-centred, which makes it better behaved than sigmoid, though it saturates just the same.
Intuition
Sigmoid has a structural annoyance: every output it produces is positive. That means the gradients flowing into the next layer's weights all share a sign, so those weights can only all increase or all decrease together — and optimisation zig-zags toward the answer instead of heading for it [3, §4.3].
Tanh is the same S-curve shifted and stretched so it straddles zero. Negative inputs give negative outputs, positive give positive, and the average activation sits near zero. That is why symmetric sigmoids are preferred: they are "more likely to produce outputs (which are inputs to the next layer) that are on average close to zero" [3, §4.4].
Formal definition
How it works
Where it's used
Why & when
Worked example
Pitfalls
Check yourself
3 questions
The questions on this page come with access. They are graded on the server, so the answers stay there until you have given yours.
On the cheat sheet: Activation functions →
Sources
- Dive into Deep Learningbook · retrieved 2026-08-31https://d2l.ai/
- Deep Learning (2016)bookhttps://www.deeplearningbook.org/
- Efficient BackProp (1998)paperhttps://doi.org/10.1007/3-540-49430-8_2