Statistics for Business: Decision Making and Analysis, 3rd Edition

16.4 Significance Versus Importance

Statistical significance does not imply that you’ve made an important or meaningful discovery, even though it may sound that way. A test produces a statistically significant result whenever we reject H₀ at the chosen α-level.

Caution

A small p-value does not mean that the difference will lead to radically greater profits than the current approach; it simply implies that the null hypothesis is inconsistent with the data.

In the example of rentals in San Francisco, suppose we had observed a much larger sample with n = 2,500. In that case, the estimated standard error of the sample mean would have been $s/\sqrt{n}=\text{\$}11.62$ We would then reject H₀ using α = 0.05 for any sample that has an average rent more than 1.645 × ...