Google Cloud Platform for Developers

Pipeline templates

In order to execute pipelines so far, we've used Maven build targets provided by the Dataflow SDK. Doing so requires a properly provisioned development environment or build pipeline in which the code can be compiled before submitting to Google Cloud. For pipelines that are executed regularly or can be generalized to a wide array of use cases, a much better option is to leverage Cloud Dataflow's pipeline templates.

Pipeline templates work by separating the pipeline development process from the execution process. Developers build and test their pipelines using the workflows we've already seen. When the pipeline is complete, a template is created using the same build tools. For Maven, this is done with the addition of a ...