Applied Data Science Using PySpark: Learn the End-to-End Predictive Model-Building Cycle

© Ramcharan Kakarla, Sundar Krishnan and Sridhar Alla 2021

R. Kakarla et al.Applied Data Science Using PySparkhttps://doi.org/10.1007/978-1-4842-6500-0_3

3. Utility Functions and Visualizations

Ramcharan Kakarla¹  , Sundar Krishnan¹ and Sridhar Alla²

(1)

Philadelphia, PA, USA

(2)

New Jersey, NJ, USA

 

In this chapter, we will dive into the utility of some of the advanced functions available in PySpark. You are encouraged to read the previous chapter and try the following operations on any dataset of your choice to improve your understanding. This chapter will focus on the windowing functions and other topics that will be useful in the creation and application of Spark programs on large datasets. We will also introduce the visualization and machine learning ...