What Is Bias?
Two Red Sox fans were discussing the finer points of baseball strategy one day while driving to Fenway Park in Boston. Burt had read a statistical study about the effectiveness of the sacrifice bunt. In this maneuver, the batter tries to advance a base-runner from first to second base by tapping the ball a few feet in front of home plate. He is willing to be thrown out at first base in exchange for helping the runner to reach second base safely. The data in the study revealed that a runner on first base scored less frequently when the batter attempted to bunt. This implied, Burt insisted, that a batter should never attempt to sacrifice. Harry disagreed. Situations in which managers called for a sacrifice bunt, he argued, were not the same as those in which batters were allowed to swing away. Somehow, Harry knew intuitively that he was right and that some deeper principle of logic was involved, but he was never able to convince his friend.
Burt was unaware that by comparing the frequency of scoring between two different sets of at-bats, he was making a biased comparison. A lower success rate observed after attempting to bunt than when “swinging away” would not necessarily mean that bunting always, or even sometimes, causes a decrease in the probability of scoring the runner. Perhaps less proficient batters often bunt, whereas stronger hitters nearly always swing away. Then the success rate of the bunters would have been lower even if they had not bunted. ...