Data Science from Scratch

Chapter 15. Multiple Regression

I don’t look at a problem and put variables in there that don’t affect it.

Bill Parcells

Although the VP is pretty impressed with your predictive model, she thinks you can do better. To that end, you’ve collected additional data: for each of your users, you know how many hours he works each day, and whether he has a PhD. You’d like to use this additional data to improve your model.

Accordingly, you hypothesize a linear model with more independent variables:

Obviously, whether a user has a PhD is not a number, but—as we mentioned in Chapter 11—we can introduce a dummy variable that equals 1 for users with PhDs and 0 for users without, after which it’s just as numeric as the other variables.

The Model

Recall that in Chapter 14 we fit a model of the form:

Now imagine that each input is not a single number but rather a vector of k numbers . The multiple regression model assumes that:

In multiple regression the vector of parameters is usually called ...