book

Scala Cookbook, 2nd Edition

Name: Scala Cookbook, 2nd Edition
Author: Alvin Alexander
ISBN: 9781492051541

by Alvin Alexander

August 2021

Intermediate to advanced

799 pages

17h 41m

English

O'Reilly Media, Inc.

Read now

Unlock full access

Preface
Conventions Used in This BookUsing Code ExamplesO’Reilly Online LearningHow to Contact Us
Command-Line Tasks
1.1. Getting Started with the Scala REPL1.2. Loading Source Code and JAR Files into the REPL1.3. Getting Started with the Ammonite REPL1.4. Compiling with scalac and Running with scala1.5. Disassembling and Decompiling Scala Code1.6. Running JAR Files with Scala and Java
Strings
2.1. Testing String Equality2.2. Creating Multiline Strings2.3. Splitting Strings2.4. Substituting Variables into Strings2.5. Formatting String Output2.6. Processing a String One Character at a Time2.7. Finding Patterns in Strings2.8. Replacing Patterns in Strings2.9. Extracting Parts of a String That Match Patterns2.10. Accessing a Character in a String2.11. Creating Your Own String Interpolator2.12. Creating Random Strings
Numbers and Dates
3.1. Parsing a Number from a String3.2. Converting Between Numeric Types (Casting)3.3. Overriding the Default Numeric Type3.4. Replacements for ++ and −−3.5. Comparing Floating-Point Numbers3.6. Handling Large Numbers3.7. Generating Random Numbers3.8. Formatting Numbers and Currency3.9. Creating New Date and Time Instances3.10. Calculating the Difference Between Two Dates3.11. Formatting Dates3.12. Parsing Strings into Dates
Control Structures
4.1. Looping over Data Structures with for4.2. Using for Loops with Multiple Counters4.3. Using a for Loop with Embedded if Statements (Guards)4.4. Creating a New Collection from an Existing Collection with for/yield4.5. Using the if Construct Like a Ternary Operator4.6. Using a Match Expression Like a switch Statement4.7. Matching Multiple Conditions with One Case Statement4.8. Assigning the Result of a Match Expression to a Variable4.9. Accessing the Value of the Default Case in a Match Expression4.10. Using Pattern Matching in Match Expressions4.11. Using Enums and Case Classes in match Expressions4.12. Adding if Expressions (Guards) to Case Statements4.13. Using a Match Expression Instead of isInstanceOf4.14. Working with a List in a Match Expression4.15. Matching One or More Exceptions with try/catch4.16. Declaring a Variable Before Using It in a try/catch/finally Block4.17. Creating Your Own Control Structures
Classes
5.1. Choosing from Domain Modeling Options5.2. Creating a Primary Constructor5.3. Controlling the Visibility of Constructor Fields5.4. Defining Auxiliary Constructors for Classes5.5. Defining a Private Primary Constructor5.6. Providing Default Values for Constructor Parameters5.7. Handling Constructor Parameters When Extending a Class5.8. Calling a Superclass Constructor5.9. Defining an equals Method (Object Equality)5.10. Preventing Accessor and Mutator Methods from Being Generated5.11. Overriding Default Accessors and Mutators5.12. Assigning a Block or Function to a (lazy) Field5.13. Setting Uninitialized var Field Types5.14. Generating Boilerplate Code with Case Classes5.15. Defining Auxiliary Constructors for Case Classes
Traits and Enums
6.1. Using a Trait as an Interface6.2. Defining Abstract Fields in Traits6.3. Using a Trait Like an Abstract Class6.4. Using Traits as Mixins6.5. Resolving Method Name Conflicts and Understanding super6.6. Marking Traits So They Can Only Be Used by Subclasses of a Certain Type6.7. Ensuring a Trait Can Only Be Added to a Type That Has a Specific Method6.8. Limiting Which Classes Can Use a Trait by Inheritance6.9. Working with Parameterized Traits6.10. Using Trait Parameters6.11. Using Traits to Create Modules6.12. How to Create Sets of Named Values with Enums6.13. Modeling Algebraic Data Types with Enums
Objects
7.1. Casting Objects7.2. Passing a Class Type with the classOf Method7.3. Creating Singletons with object7.4. Creating Static Members with Companion Objects7.5. Using apply Methods in Objects as Constructors7.6. Implementing a Static Factory with apply7.7. Reifying Traits as Objects7.8. Implementing Pattern Matching with unapply
Methods
8.1. Controlling Method Scope (Access Modifiers)8.2. Calling a Method on a Superclass or Trait8.3. Using Parameter Names When Calling a Method8.4. Setting Default Values for Method Parameters8.5. Creating Methods That Take Variable-Argument Fields8.6. Forcing Callers to Leave Parentheses Off Accessor Methods8.7. Declaring That a Method Can Throw an Exception8.8. Supporting a Fluent Style of Programming8.9. Adding New Methods to Closed Classes with Extension Methods
Packaging and Imports
9.1. Packaging with the Curly Braces Style Notation9.2. Importing One or More Members9.3. Renaming Members on Import9.4. Hiding a Class During the Import Process9.5. Importing Static Members9.6. Using Import Statements Anywhere9.7. Importing Givens

Functional Programming
10.1. Using Function Literals (Anonymous Functions)10.2. Passing Functions Around as Variables10.3. Defining a Method That Accepts a Simple Function Parameter10.4. Declaring More Complex Higher-Order Functions10.5. Using Partially Applied Functions10.6. Creating a Method That Returns a Function10.7. Creating Partial Functions10.8. Implementing Functional Error Handling10.9. Real-World Example: Passing Functions Around in an Algorithm10.10. Real-World Example: Functional Domain Modeling
Collections: Introduction
11.1. Choosing a Collections Class11.2. Understanding the Performance of Collections11.3. Understanding Mutable Variables with Immutable Collections11.4. Creating a Lazy View on a Collection
Collections: Common Sequence Classes
12.1. Making Vector Your Go-To Immutable Sequence12.2. Creating and Populating a List12.3. Adding Elements to a List12.4. Deleting Elements from a List (or ListBuffer)12.5. Creating a Mutable List with ListBuffer12.6. Using LazyList, a Lazy Version of a List12.7. Making ArrayBuffer Your Go-To Mutable Sequence12.8. Deleting Array and ArrayBuffer Elements12.9. Creating and Updating an Array12.10. Creating Multidimensional Arrays12.11. Sorting Arrays
Collections: Common Sequence Methods
13.1. Choosing a Collection Method to Solve a Problem13.2. Looping Over a Collection with foreach13.3. Using Iterators13.4. Using zipWithIndex or zip to Create Loop Counters13.5. Transforming One Collection to Another with map13.6. Flattening a List of Lists with flatten13.7. Using filter to Filter a Collection13.8. Extracting a Sequence of Elements from a Collection13.9. Splitting Sequences into Subsets13.10. Walking Through a Collection with the reduce and fold Methods13.11. Finding the Unique Elements in a Sequence13.12. Merging Sequential Collections13.13. Randomizing a Sequence13.14. Sorting a Collection13.15. Converting a Collection to a String with mkString and addString
Collections: Using Maps
14.1. Creating and Using Maps14.2. Choosing a Map Implementation14.3. Adding, Updating, and Removing Immutable Map Elements14.4. Adding, Updating, and Removing Elements in Mutable Maps14.5. Accessing Map Values (Without Exceptions)14.6. Testing for the Existence of a Key or Value in a Map14.7. Getting the Keys or Values from a Map14.8. Finding the Largest (or Smallest) Key or Value in a Map14.9. Traversing a Map14.10. Sorting an Existing Map by Key or Value14.11. Filtering a Map
Collections: Tuple, Range, Set, Stack, and Queue
15.1. Creating Heterogeneous Lists with Tuples15.2. Creating Ranges15.3. Creating a Set and Adding Elements to It15.4. Deleting Elements from Sets15.5. Storing Values in a Set in Sorted Order15.6. Creating and Using a Stack15.7. Creating and Using a Queue
Files and Processes
16.1. Reading Text Files16.2. Writing Text Files16.3. Reading and Writing Binary Files16.4. Pretending That a String Is a File16.5. Serializing and Deserializing Objects to Files16.6. Listing Files in a Directory16.7. Executing External Commands16.8. Executing External Commands and Reading Their STDOUT16.9. Handling Both STDOUT and STDERR of Commands16.10. Building a Pipeline of External Commands
Building Projects with sbt
17.1. Creating a Project Directory Structure for sbt17.2. Building Projects with the sbt Command17.3. Understanding build.sbt Syntax Styles17.4. Compiling, Running, and Packaging a Scala Project17.5. Understanding Other sbt Commands17.6. Continuous Compiling and Testing17.7. Managing Dependencies with sbt17.8. Controlling Which Version of a Managed Dependency Is Used17.9. Generating Project API Documentation17.10. Specifying a Main Class to Run with sbt17.11. Deploying a Single Executable JAR File17.12. Publishing Your Library
Concurrency with Scala Futures and Akka Actors
18.1. Creating a Future18.2. Using Callback and Transformation Methods with Futures18.3. Writing Methods That Return Futures18.4. Running Multiple Futures in Parallel18.5. Creating OOP-Style Actors18.6. Creating FP-Style Actors18.7. Sending Messages to Actors18.8. Creating Actors That Have Multiple States (FSM)
Play Framework and Web Services
19.1. Creating a Play Framework Project19.2. Creating a New Play Framework Endpoint19.3. Returning JSON from a GET Request with Play19.4. Serializing a Scala Object to a JSON String19.5. Deserializing JSON into a Scala Object19.6. Using the Play JSON Library Outside of the Play Framework19.7. Using the sttp HTTP Client
Apache Spark
20.1. Getting Started with Spark20.2. Reading a File into a Spark RDD20.3. Reading a CSV File into a Spark RDD20.4. Using Spark Like a Database with DataFrames20.5. Reading Data Files into a Spark DataFrame20.6. Using Spark SQL Queries Against Multiple Files20.7. Creating a Spark Batch Application
Scala.js, GraalVM, and jpackage
21.1. Getting Started with Scala.js21.2. Responding to Events with Scala.js21.3. Building Single-Page Applications with Scala.js21.4. Building Native Executables with GraalVM21.5. Bundling Your Application with jpackage
Integrating Scala with Java
22.1. Using Java Collections in Scala22.2. Using Scala Collections in Java22.3. Using Java Optional Values in Scala22.4. Using Scala Option Values in Java22.5. Using Scala Traits in Java22.6. Using Java Interfaces in Scala22.7. Adding Exception Annotations to Scala Methods22.8. Annotating varargs Methods to Work with Java22.9. Using @SerialVersionUID and Other Annotations
Types
23.1. Creating a Method That Takes a Simple Generic Type23.2. Creating Classes That Use Simple Generic Types23.3. Making Immutable Generic Parameters Covariant23.4. Creating a Class Whose Generic Elements Can Be Mutated23.5. Creating a Class Whose Parameters Implement a Base Type23.6. Using Duck Typing (Structural Types)23.7. Creating Meaningful Type Names with Opaque Types23.8. Using Term Inference with given and using23.9. Simulating Dynamic Typing with Union Types23.10. Declaring That a Value Is a Combination of Types23.11. Controlling How Classes Can Be Compared with Multiversal Equality23.12. Limiting Equality Comparisons with the CanEqual Typeclass
Best Practices
24.1. Writing Pure Functions24.2. Using Immutable Variables and Collections24.3. Writing Expressions (Instead of Statements)24.4. Using Match Expressions and Pattern Matching24.5. Eliminating null Values from Your Code24.6. Using Scala’s Error-Handling Types (Option, Try, and Either)24.7. Building Modular Systems24.8. Handling Option Values with Higher-Order Functions
Index

Content preview from Scala Cookbook, 2nd Edition

Chapter 20. Apache Spark

This chapter demonstrates recipes for Apache Spark, a unified data analytics engine for large-scale data processing.

The Spark website describes Spark as a “unified analytics engine for large-scale data processing.” This means that it’s a big data framework that lets you analyze your data with different techniques—such as treating the data as a spreadsheet or as a database—and runs on distributed clusters. You can use Spark to analyze datasets that are so large that they span thousands of computers.

While Spark is designed to work with enormous datasets on clusters of computers, a great thing about it is that you can learn how to use Spark on your own computer with just a few example files.

Spark 3.1.1

The examples in this chapter use Spark 3.1.1, which was released in March 2021 and is the latest release at the time of this writing. Spark currently works only with Scala 2.12, so the examples in this chapter also use Scala 2.12. However, because working with Spark generally involves using collections methods like map and filter, or SQL queries, you’ll barely notice the difference between Scala 2 and Scala 3 in these examples.

The recipes in this chapter show how to work with Spark on your own computer, while demonstrating the key concepts that work on datasets that span thousands of computers. Recipe 20.1 demonstrates how to get started with Spark and digs into one of its fundamental concepts, the Resilient Distributed Dataset, or RDD. An RDD lets you ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 9781492051534Errata Page

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Scala Cookbook, 2nd Edition

by Alvin Alexander

Chapter 20. Apache Spark

Spark 3.1.1

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.