Data Analysis with R, Second Edition - Second Edition

Errors in NHST

Certainty is a card rarely seen in the deck of a data analyst. As we make judgments and inferences based on probabilities, mistakes happen. In particular, there are two types of mistakes that are possible in NHST: Type I errors and Type II errors:

• A Type I error is when a hypothesis test concludes that there is an effect (rejects the null hypothesis) when, in reality, n o such effect exists
• A Type II error occurs when we fail to detect a real effect in the world and fail to reject the null hypothesis even if it is false

Check the following table for errors encountered in the coin example:

Coin type

Failure to reject null hypothesis (conclude no detectable effect)

Reject the null hypothesis (conclude that there is ...