May 2018
Intermediate to advanced
576 pages
14h 42m
English
These two activations are very similar but with an important difference. Let's start defining them:

The corresponding plots are shown in the following diagram:

A sigmoid σ(x) is bounded between 0 and 1, with two asymptotes (σ(x) → 0 when x → -∞ and σ(x) → 1 when x → ∞). Similarly, the hyperbolic tangent (tanh) is bounded between -1 and 1 with two asymptotes corresponding to the extreme values. Analyzing the two plots, we can discover that both functions are almost linear in ...
Read now
Unlock full access