R Statistics Cookbook

Variable selection

A fundamental question when doing linear regression is how to choose the best subset of variables that we have already included. Every variable that is added to a model changes the standard errors of the other variables already included. Consequently, the p-values also change, and the order is relevant. This happens because in general the variables are correlated, causing the coefficients' covariance matrix to change (hence changing the standard errors). Sandwich estimators use a different formula for the standard errors. Note the Ω which is the new element here. This matrix is estimated by the sandwich package. This formula also explicits why this is called the sandwich method (the Ω gets sandwiched between two equal ...