In this section, we will learn how to save data in plain text format. The following topics will be covered:
- Saving data in plain text format
- Loading plain text data
- Testing
We will save our data in plain text format and investigate how to save it into the Spark directory. We will then load the plain text data, and then test and save it to check whether we can yield the same results code. This is our SavePlainText.scala file:
package com.tomekl007.chapter_4import java.io.Fileimport com.tomekl007.UserTransactionimport org.apache.spark.sql.SparkSessionimport org.apache.spark.{Partitioner, SparkContext}import org.scalatest.{BeforeAndAfterEach, FunSuite}import org.scalatest.Matchers._import scala.reflect.io.Path ...
