Mastering Machine Learning with Spark 2.x

Deploying a pattern mining application

The example developed in the last section was an interesting playground to apply the algorithms we have carefully laid out throughout the chapter, but we have to recognize the fact that we were just handed the data. At the time of writing this book, it was often part of the culture in building data products to draw a line in the sand between data science and data engineering at pretty much exactly this point, that is, between real-time data collection and aggregation, and (often offline) analysis of data, followed up by feeding back reports of the insights gained into the production system. While this approach has its value, there are certain drawbacks to it as well. By not taking the full picture into ...