Scala and Spark for Big Data Analytics

Covariance

Covariance is a measure of the joint variability of two random variables. If the greater values of one variable mainly corresponds with the greater values of the other variable, and the same holds for the lesser values, then the variables tend to show similar behavior and the covariance is positive. If the opposite is true, and the greater values of one variable correspond with the lesser values of the other variable, then the covariance is negative.

The covar API has several implementations, as follows. The exact API used depends on the specific use case.

def covar_pop(columnName1: String, columnName2: String): ColumnAggregate function: returns the population covariance for two columns.def covar_pop(column1: Column, column2: Column): ...