Practical Apache Spark: Using the Scala API

© Subhashini Chellappan, Dharanitharan Ganesan 2018

Subhashini Chellappan and Dharanitharan GanesanPractical Apache Sparkhttps://doi.org/10.1007/978-1-4842-3652-9_9

9. Working with SparkR

Subhashini Chellappan¹  and Dharanitharan Ganesan²

(1)

Bangalore, India

(2)

Krishnagiri, Tamil Nadu, India

 

In the previous chapter, we discussed the fundamental concepts of Spark MLlib. We also discussed the machine learning algorithms with implementation.

In this chapter, we are going to discuss how to work with the SparkR component. We focus on the following topics:

• Introduction to SparkR.

• Starting SparkR from RStudio.

• Creating a SparkDataFrame.

• SparkDataFrame operations.

• Applying user-defined functions.

• Running SQL queries.

Introduction to SparkR

SparkR is an R package that ...