The R Book

Polynomial Regression

The relationship between y and x often turns out not to be a straight line. However, Occam's razor requires that we fit a straight-line model unless a non-linear relationship is significantly better at describing the data. So this begs the question: how do we assess the significance of departures from linearity? One of the simplest ways is to use polynomial regression.

The idea of polynomial regression is straightforward. As before, we have just one continuous explanatory variable, x, but we can fit higher powers of x, such as x² and x³, to the model in addition to x to explain curvature in the relationship between y and x. It is useful to experiment with the kinds of curves that can be generated with very simple models. Even if we restrict ourselves to the inclusion of a quadratic term, x², there are many curves we can describe, depending upon the signs of the linear and quadratic terms:

In the top left-hand panel, there is a curve with positive but declining slope, with no hint of a hump (y = 4 + 2x − 0.1x²). The top right-hand graph shows a curve with a clear maximum (y = 4 + 2x − 0.2x²), and at bottom left we have a curve with a clear minimum (y = 12 − 4x + 0.35x²). The bottom right-hand curve shows a positive association between y and x with the slope increasing as x increases (y = 4 + 0.5x + 0.1x²). So you can see that a simple quadratic model with three parameters (an intercept, a slope for x, and a slope for x²) is capable of describing a wide range ...