Agile Data Science

Chapter 8. Making Predictions

Now that we have interactive reports exposing different aspects of our data, we’re ready to make our first prediction. This forms our fourth agile sprint (Figure 8-1). When making predictions, we take what we know about the past and project what will happen in the future, simultaneously transitioning from batch processing of historical data to real-time classification of the present to predict the future. We’ll start simply, moving on to driving real actions in the next chapter.

Figure 8-1. Level 4: making predictions

Code examples for this chapter are available at https://github.com/rjurney/Agile_Data_Code/tree/master/ch08. Clone the repository and follow along!

git clone https://github.com/rjurney/Agile_Data_Code.git

Predicting Response Rates to Emails

When I click around in our application and look at the charts showing how often someone emails by hour of the day, I wonder if we can infer from this data when someone is most likely to reply. This is why we created charts and reports in the first place—to guide us as we climb the data-value pyramid.

In this chapter, we will predict whether a recipient will respond to a given email using some of the entities we’ve extracted from our inbox. In the next chapter, we’ll use this inference to enable a new kind of action.

We’re going to walk from simple frequencies to real insight one table at a time, just as we did ...