Data Science: The Hard Parts

Chapter 4. 2×2 Designs

Some years ago, when I was starting my career in data science, a consulting firm came to the office and started sketching these extremely simplified views of our business. My immediate reaction was to dismiss these sketches as a trick in their sales-driven bag. Today I embrace them for communication and storytelling purposes, as well as useful aids to simplify a complex business.

I believe that a natural growth path in data science (DS) is to go from making things overly complex to doing smart simplification. By smart I mean what Einstein expressed when saying you should aim at making “everything as simple as possible, but not simpler.” The beauty of this quote is that it shows how difficult it is to achieve this. In this chapter, I’ll make a case for using a tool designed for the specific purpose of simplifying a complex world.

The Case for Simplification

You may find it ironic that I make a case for simplification in the age of big data, computational power, and sophisticated predictive algorithms. These tools allow you to navigate the ever-increasing volumes of data and thus have undoubtedly improved data scientists’ productivity, but they don’t really simplify the world or the business.

Let’s stop for a second on this last thought: if more data means more complexity, then data scientists are now definitely capable of making sense of more complexity. Nonetheless, the fact that you can make projections of high-dimensional data onto lower-dimensional scores ...