Scala and Spark for Big Data Analytics

Last

Gets the last record in the RelationalGroupeddataset.

The last API has several implementations, as follows. The exact API used depends on the specific use case:

def last(columnName: String): ColumnAggregate function: returns the last value of the column in a group.def last(e: Column): ColumnAggregate function: returns the last value in a group.def last(columnName: String, ignoreNulls: Boolean): ColumnAggregate function: returns the last value of the column in a group.def last(e: Column, ignoreNulls: Boolean): ColumnAggregate function: returns the last value in a group.

Let's look at an example of invoking last on the DataFrame to output the last row.

import org.apache.spark.sql.functions._scala> statesPopulationDF.select(last("State")).show ...