Mastering Computer Vision with TensorFlow 2.x

Activation

The activation layer adds nonlinearity to the neural network. This is critical as images and features within an image are highly non-linear problems, and most other functions within CNNs (Conv2D, pooling, fully connected layers, and so on) generate only linear transformations. The activation function generates the non-linearity while mapping input values to its ranges. Without the activation function, no matter how many layers are added, the final result will still be linear.

Many types of activation functions are used, but the most common ones are as follows:

• Sigmoid
• Tanh
• ReLU

The preceding activation functions can be seen in the following graph:

Each of the activation functions shows non-linear behavior, with Sigmoid and ...