Deep Learning Essentials

Sigmoid or logistic function

A sigmoid function has a distinctive S shape and it is a differentiable real function for any real input value. Its range is between 0 and 1. It is an activation function in the following form:

Its first derivative, which is used during backpropagation of the training step, has the following form:

The implementation is as follows:

def sigmoid(x):    return tf.div(tf.constant(1.0),                  tf.add(tf.constant(1.0), tf.exp(tf.neg(x))))

The derivative of a sigmoid function is as follows:

def sigmoidprime(x): return tf.multiply(sigmoid(x), ...