O'Reilly logo

Illuminating Statistical Analysis Using Scenarios and Simulations by Jeffrey E. Kottemann

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

17Brief on Type I Versus Type II Errors

img

Note: You may want to reread lesson 1 at this point.

Imagine testing many, many different coins, where their balance is unknown and can be different from coin to coin. Type I error is mistakenly rejecting the null hypothesis when it is actually true. The probability threshold we use for type I error is alpha. When testing a coin for fairness, type I error would be judging a fair coin to be unfair. Type I error is relatively straightforward to control in practice because we set the alpha-level and we are only concerned with the fate of the fair coins.

Type II error is mistakenly not rejecting the null hypothesis when it is actually false. With a coin, that would be judging an unfair coin to be fair. The probability for type II error is called beta. We can easily determine beta if we want to test for a certain specific degree of unfairness such as a 30% chance of heads. However, it is difficult to estimate beta in general because then we are concerned with all the variously unbalanced unfair coins. You can imagine that for unfair coins that have a 51% chance of coming up heads, then we'll make many type II errors. On the other hand, for unfair coins that have a 1% chance of coming up heads, then we'll make very few type II errors. What level of unfairness are we dealing with for each coin? We often don't know for sure. We have to make specific ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required