Scala and Spark for Big Data Analytics

DropWhile

dropWhile is used to omit a set of elements till a predicate is satisfied. It can be defined formally as follows:

def dropWhile(p: (A) ⇒ Boolean): Traversable[A]  

Let's see an example as follows:

//Drop values till reaching the border between numbers that are greater than 5 and less than 5scala> List(2,3,4,9,10,11) dropWhile(x => x <5)res1: List[Int] = List(9, 10, 11)

In Scala, if you want to use your User Defined Functions (UDF) such that it takes a function as an argument in the nested list and combines the output back together, flatMap() is a perfect candidate. We will see examples of using flatMap() in the next section.