Hands-On Mathematics for Deep Learning

Hyperbolic tangent

Another activation function used instead of the sigmoid is the hyperbolic tangent (tanh). It is written as follows:

The function looks as follows:

The tanh function squashes all the output values into the (-1, 1) range. Its derivative is as follows:

The derivative looks as follows:

From the preceding graph you can tell that ...