Hands-On Deep Learning with R

Exploring the sigmoid function

The sigmoid function is the classic S-shaped function. This function works especially well for logistic regression tasks. While most of the results will be classified by the tails on either side of the curve, there is an area in the middle for capturing uncertainty about some of the data. The drawback of this shape is that the gradient is almost zero at the extremes, so the model may not be able to continue to learn as the points get towards either side.

The sigmoid function also contains a derivative value, which means that we can use this function along with backpropagation to update the weights after the variables pass through additional layers. We will explore backpropagation more in the final parts of this ...