In Chapter 3 we discussed permutation methods for two-sample tests and we permuted the residuals to perform that test. We also showed that permutations of residuals was equivalent to permutations of observations in that two-sample test. Both of the methods are valid, so there is little controversy about the permutation method.
In contrast, when different permutation methods are used to test a coefficient in a linear regression model the results are not equivalent and it is not clear which method is superior. Some of the methods that have been proposed are:
We will describe the first three of these methods in some detail in this chapter. For details concerning the last method see Anderson and Legendre (1999).
Usually we are not able to tabulate the full permutation distribution, so we take a sample of permutations from a population of all possible permutations. We could use sampling with replacement or sampling without replacement. Sampling without replacement could slightly reduce the variability in the p-value, but ...