Apache Spark Quick Start Guide

Serializing and compressing 

Whenever we use custom data types in our Spark application, we should serialize them. Java serialization is the default serialization but, changing the serialization format to Kryo can increase the performance. Objects serialized by Java serializer are often slow and larger in size as compared to the Kryo serializer. You can change the spark.serializer property to org.apache.spark.serializer.KryoSerializer to set the Kryo serializer. You would require your class to register with the Kryo and you can be strict about class registration. In the following example, we first mandate the class registration by setting the flag spark.kryo.registrationRequired to true. We then register two of our classes in Scala:

//Scala ...