Analytics the Right Way

Chapter 9Pitfalls and Solutions for Scientific Evidence

There aren’t many things more disappointing than being sure that something is going to work out, and then realizing…it won’t. When evidence is presented as “scientific,” it carries with it an air of certainty. Cue the bold declaration: “Of course my hypothesis is validated, and we should pursue its implications immediately…I used science!” Ultimately, when that scientific evidence generates conclusions that are proven wrong―but which were trotted out by someone as if they were dogmatic certainties―the faith others have in the analyses produced by that person can be seriously degraded. We may get a few points docked for our analysis being wrong, but we lose the farm when we’re just plain foolish with the way we went about it.

The point of this chapter is to arm you with an understanding of how scientific evidence should be generated (and where people go wrong), so that you know when to put your stock in an analysis. Scientific evidence is not a silver bullet, and as we’ve stressed throughout this book, it is impossible to achieve 100% certainty about really anything. The problem really sets in when we develop unfounded certainty in something because we have produced faulty evidence, which we accordingly overweight in our consideration of a decision. It’s like charging headlong off a cliff for a bungee jump and then realizing ...