Analytics for the Internet of Things (IoT)

Using Spark for IoT data processing

Using our conceptual diagram from earlier in the chapter, Spark fits into both the analysis interaction and the execution engine areas. You can write code directly on the core Spark objects and also make use of the programming modules that come along with it. The following image helps to visualize the Spark module stack:

Spark Module Stack. Source: Apache Spark

Spark SQL is included with the SparkSession when it is created and allows you to write standard SQL that is interpreted into Spark actions and executed (using a DAG, of course). You can use it intermixed with other Spark code. This allows you to leverage ...