Advanced Deep Learning with Python

Neurons

The preceding definition is a bird's-eye view of a NN. Now, let's discuss the basic building blocks of a NN, namely the neurons (or units). Units are mathematical functions that can be defined as follows:

Here, we have the following:

• y is the unit output (single value).
• f is the non-linear differentiable activation function. The activation function is the source of non-linearity in a NN—if the NN was entirely linear, it would only be able to approximate other linear functions.
• The argument of the activation function is the weighted sum (with weights w_i) of all the unit inputs x_i (n total inputs) and the bias weight b. The inputs ...