Python Machine Learning By Example - Second Edition

Training a logistic regression model with regularization

As we briefly mentioned in the previous section, the penalty parameter in the logistic regression SGDClassifier is related to model regularization. There are two basic forms of regularization, L1 (also called Lasso) and L2 (also called ridge). In either way, the regularization is an additional term on top on the original cost function:

Here, α is the constant that multiplies the regularization term, and q is either 1 or 2 representing L1 or L2 regularization where the following applies:

Training a logistic regression model is a process of reducing the cost as a function of weights ...