Python Machine Learning By Example

Logistic regression

It is probably the most widely used classification algorithm, and the first algorithm a machine learning practitioner usually tries given a classification problem. It performs well when data is linearly separable or approximately linearly separable. Even if it is not linearly separable, we can if possible, convert the linearly non-separable features into separable ones and apply logistic regression afterwards (see the following example). Also logistic regression is extremely scalable to large datasets with SGD optimization, which makes it efficient in solving big data problems. Plus, it makes online learning feasible.

Although logistic regression is a low bias, high variance algorithm, we overcome the potential overfitting ...