Mastering Machine Learning with Spark 2.x

Preface

Big data – that was our motivation to explore the world of machine learning with Spark a couple of years ago. We wanted to build machine learning applications that would leverag models trained on large amounts of data, but the beginning was not easy. Spark was still evolving, it did not contain a powerful machine learning library, and we were still trying to figure out what it means to build a machine learning application.

But, step by step, we started to explore different corners of the Spark ecosystem and followed Spark’s evolution. For us, the crucial part was a powerful machine learning library, which would provide features such as R or Python libraries did. This was an easy task for us, since we are actively involved in the development ...