O'Reilly logo

Practical Predictive Analytics by Ralph Winters

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Reading the data

In the last chapter, we saved the out_sd to an external parquet file. In the real world, you will be faced with analyzing multiple data sources. Often, these data sources will have similar schemas but will differ by the time period that they were written.

For example, log files can be archived in different directories, categorized by date, and you as an analyst will need to read in multiple files and concatenate them together.

So although each individual file may be small, when aggregated together, they will yield a much larger file.

Let's pretend that we will be performing some machine learning on several files that have been written over the past two days. For illustration purposes, we will use the same file for the two ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required