Scala Programming Projects

Writing transactions in Parquet

Add the following import and function declaration in BatchProducer:

import java.net.URIdef unsafeSave(transactions: Dataset[Transaction], path: URI): Unit = ???

Let's have a look in more detail:

• Writing to a file is a side effect; this is why we prefixed our function with unsafe. As functional programmers, we strive to control side effects, and it is a good practice to name any side effecting function explicitly. We will see in the next section how to use the IO Monad to push this side effect to the boundaries of our application.

• We use java.net.URI to pass the path to the directory where our files will be written. This makes sure that the path we pass to the function is really a path. As usual, we try ...