Doing Data Science

Chapter 1. Introduction: What Is Data Science?

Over the past few years, there’s been a lot of hype in the media about “data science” and “Big Data.” A reasonable first reaction to all of this might be some combination of skepticism and confusion; indeed we, Cathy and Rachel, had that exact reaction.

And we let ourselves indulge in our bewilderment for a while, first separately, and then, once we met, together over many Wednesday morning breakfasts. But we couldn’t get rid of a nagging feeling that there was something real there, perhaps something deep and profound representing a paradigm shift in our culture around data. Perhaps, we considered, it’s even a paradigm shift that plays to our strengths. Instead of ignoring it, we decided to explore it more.

But before we go into that, let’s first delve into what struck us as confusing and vague—perhaps you’ve had similar inclinations. After that we’ll explain what made us get past our own concerns, to the point where Rachel created a course on data science at Columbia University, Cathy blogged the course, and you’re now reading a book based on it.

Big Data and Data Science Hype

Let’s get this out of the way right off the bat, because many of you are likely skeptical of data science already for many of the reasons we were. We want to address this up front to let you know: we’re right there with you. If you’re a skeptic too, it probably means you have something useful to contribute to making data science into a more legitimate field ...