Hands-On Deep Learning with Go

Tanh

It can also be helpful to have a steeper gradient during training; as such, we can use the tanh function instead of the Sigmoid function, as shown in the following code:

func tanh(x){  return 2 * (1 + Exp(-2*x)) - 1}

We get the following output:

The tanh function has another useful property: its slope is much steeper than the Sigmoid function; this helps networks with tanh activation functions to descend the gradient faster when adjusting weights. The output for both functions is plotted in the following output: