Skip to Main Content
Modern Scala Projects
book

Modern Scala Projects

by Ilango gurusamy
July 2018
Intermediate to advanced content levelIntermediate to advanced
334 pages
8h 20m
English
Packt Publishing
Content preview from Modern Scala Projects

Building the AirlineWrapper Scala trait

The AirlineWrapper contains code to create a SparkSession instance called session. It also declares case classes to represent our flights dataset. 

Let's create the trait definition first:

trait AirlineWrapper {  }

The entry point to programming is as follows: The first thing we do in the trait is to declare a lazy val called session. This is where we lazily create an instance of SparkSession. Lazily implies that the val is only executed when it is encountered the first time around. The session is our entry point to programming Spark with the DataSet and DataFrame API is SparkSession:

lazy val session = { SparkSession.builder()..getOrCreate() }

In the following code snippet, CarrierCode is an identification ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Start your free trial

You might also like

Scala Programming Projects

Scala Programming Projects

Mikael Valot, Nicolas Jorand

Publisher Resources

ISBN: 9781788624114Supplemental Content