Mastering Machine Learning with scikit-learn - Second Edition

Activation functions

The perceptron classifies instances by processing a linear combination of the features and the model parameters using an activation function, as shown in the following equation:

Here, w_i are the model's parameters, b is a constant bias term, and ϕ is the activation function. The linear combination of the parameters and inputs is sometimes called preactivation. Several different activation functions are commonly used. Rosenblatt's original perceptron used the Heaviside step function. Also called the unit step function, the Heaviside step function is shown in the following equation, where x is the weighted combination of ...