Artificial Intelligence for Big Data

Activation functions

Without an activation function, the output will be a linear function of the input values. A linear function is a straight line equation or a polynomial equation of the first degree. A linear equation represents the simplest form of a mathematical model and is not representative of real-world scenarios. It cannot map the correlations within complex datasets. Without an activation function, a neural network will have very limited capability to learn and model unstructured datasets such as images and videos. The difference between a linear and nonlinear function is illustrated in the following diagram:

Figure 4.6: Linear ...