Chapter 9: Building Your Data Pipeline

In the previous chapter, you understood the example business goal of improving user experience by recommending flights that have a higher on-time probability. You have worked with the business subject matter expert (SME) to understand the available data. In this chapter, you will see how the platform assists you in harvesting and processing data from a variety of sources. You will see how on-demand Spark clusters can be created and how workloads could be isolated in a shared environment using the platform. New flights data may be available on a frequent basis and you will see how the platform enables you to automate the execution of your data pipeline.

In this chapter, you will learn about the following ...

Get Machine Learning on Kubernetes now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.