Scala and Spark for Big Data Analytics

Tuning the data structures

The first way to reduce extra memory usage is to avoid some features in the Java data structure that impose extra overheads. For example, pointer-based data structures and wrapper objects contribute to nontrivial overheads. To tune your source code with a better data structure, we provide some suggestions here, which can be useful.

First, design your data structures such that you use arrays of objects and primitive types more. Thus, this also suggests using standard Java or Scala collection classes like Set, List, Queue, ArrayList, Vector, LinkedList, PriorityQueue, HashSet, LinkedHashSet, and TreeSet more frequently.

Second, when possible, avoid using nested structures with a lot of small objects and pointers so ...