Introducing .NET for Apache Spark: Distributed Processing for Massive Datasets

© Ed Elliott 2021

E. ElliottIntroducing .NET for Apache Sparkhttps://doi.org/10.1007/978-1-4842-6992-3_3

3. Programming with .NET for Apache Spark

Ed Elliott¹  

(1)

Sussex, UK

 

In this chapter, we will be writing our first .NET for Apache Spark application that we can execute and even debug in our favorite .NET IDE. We will cover what it is we need to do in our project and then what we need to do so that we can execute the program using Apache Spark.

Once we have got our first program running, we will move on to look at how to convert from existing PySpark examples and highlight some Scala features that we need to remember when converting from Scala into .NET. The reason for translating from Python and Scala is that they are the most commonly used languages ...