Neural Networks with Keras Cookbook

From convolution to activation

In a traditional NN, a hidden layer not only multiplies the input values by the weights, but also applies a non-linearity to the data, that is, it passes the values through an activation function.

A similar activity happens in a typical CNN too, where the convolution is passed through an activation function. CNN supports the traditional activations functions we have seen so far: sigmoid, ReLU, tanh, and leaky ReLU.

For the preceding output, we can see that the output remains the same when passed through a ReLU activation function, as all the numbers are positive.