Building Google Cloud Platform Solutions

Executing pipelines locally

As stated earlier in this chapter, Cloud Dataflow is just one of several Apache Beam runners. Another commonly used runner is Direct Runner, which executes the pipeline locally. This is extremely useful for development tasks such as debugging and writing unit tests. We can try out this pipeline locally by first including the runner on the project's class path:

<dependency>    <groupId>org.apache.beam</groupId>    <artifactId>beam-runners-direct-java</artifactId>    <version>${beam.version}</version></dependency>

Next, using a machine with both Java 8 and Apache Maven (such as the Cloud Shell) execute the pipeline with the following command within chapter_15/example_01 of this book's source code:

mvn compile exec:java \ ...