Python Machine Learning By Example - Second Edition

Polynomial features

If we have two features, a and b, we can suspect that there's a polynomial relation, such as a2 + ab + b2. We can consider each term in the sum to be a feature—in this example, we have three features. The product ab in the middle is called an interaction. An interaction doesn't have to be a product—although this is the most common choice—it can also be a sum, a difference, or a ratio. If we're using a ratio to avoid dividing by zero, we should add a small constant to the divisor and dividend.

The number of features and the order of the polynomial for a polynomial relation aren't limited. However, if we follow Occam's razor, we should avoid higher-order polynomials and interactions of many features. In practice, complex ...