Practical Predictive Analytics

P-values and effect size

P-values have been the traditional way of performing hypothesis testing on output statistics, but have come under criticism in recent year as not being the appropriate measure to use, in many cases.

Datasets are much larger than they used to be. A very large dataset with a large number of variables can always show a significant p-value, even when none exist.

There are also a large number of different kinds of algorithms as well than can produce p-values, and there is a good chance you can pick one which will give you the p-value you want.

There are different p-value cut-off levels that an analyst can use to force significance.

The more dangerous situation is when you can select or alter the data to affect the p-value. ...