Chapter 6: Using Your Preferred Language with Portability
In the previous chapters, we focused on the Java SDK – or various Java SDK-based DSLs – but what if we want to implement our data transformation logic in a completely different language, such as Python or Go? One of the main goals of Apache Beam is portability. We have already seen the portability of pipelines between different Runners and between batch and streaming semantics. In this chapter, we will explore the last aspect of portability – portability between SDKs.
We will outline how the portability layer works (Apache Beam often calls it the Fn API – pronounced Fun API) so that the result is portable. The desired goal is to enable Runners so that they don't have to understand the ...
Get Building Big Data Pipelines with Apache Beam now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
