Building Machine Learning Pipelines

Chapter 10. Advanced TensorFlow Extended

With the previous two chapters on model deployments, we completed our overview of individual pipeline components. Before we take a deep dive into orchestrating these pipeline components, we want to pause and introduce advanced concepts of TFX in this chapter.

With the pipeline components we have introduced so far, we can create machine learning pipelines for most problems. However, sometimes we need to build our own TFX component or more complex pipeline graphs. Therefore, in this chapter, we will focus on how to build custom TFX components. We introduce the topic with a custom ingestion component that ingests images directly for computer vision ML pipelines. Furthermore, we will introduce advanced concepts of pipeline structures: generating two models simultaneously (e.g., for deployments with TensorFlow Serving and TFLite), as well as adding a human reviewer into the pipeline workflow.

Ongoing Developments

At the time of this writing, some of the concepts we are introducing are still under development and, therefore, might be subject to future updates. We have done our best to update code examples with changes to the TFX functionality throughout the production of this publication, and all examples work with TFX 0.22. Updates to the TFX APIs can be found in the TFX documentation.

Advanced Pipeline Concepts

In this section, we will discuss three additional concepts to advance your pipeline setups. So far, all the pipeline concepts we’ve ...