O'Reilly logo

Apache Spark 2.x Machine Learning Cookbook by Shuen Mei, Broderick Hall, Meenakshi Rajendran, Siamak Amirghodsi

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

How to do it...

  1. Start a new project in IntelliJ or in an IDE of your choice. Make sure the necessary JAR files are included.
  1. Set up the path using the package directive: package spark.ml.cookbook.chapter9.
  1. Import the necessary packages.

The scala.util.control.Breaks will allow us to break out of the program. We use this during the debugging phase only when the program fails to converge or gets stuck in a never ending process (for example, when the step size is too large).

import scala.collection.mutable.ArrayBufferimport scala.util.control.Breaks._
  1. This step defines the actual quadratic function that we are trying to minimize:
def quadratic_function_itself(x:Double):Double = {// the function being differentiated// f(x) = 2x^2 - 8x ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required