Effective Data Science Infrastructure

3 Introducing Metaflow

This chapter covers

• Defining a workflow in Metaflow that accepts input data and produces useful outputs
• Optimizing the performance of workflows with parallel computation on a single instance
• Analyzing the results of workflows in notebooks
• Developing a simple end-to-end application in Metaflow

You are probably anxious to roll up your sleeves and start hacking actual code, now that we have a development environment set up. In this chapter, you will learn the basics of developing data science applications using Metaflow, a framework that shows how different layers of the infrastructure stack can work together seamlessly.

The development environment, which we discussed in the previous chapter, determines how the data scientist ...