Hands-On Deep Learning with Apache Spark

Memory management

In Chapter 7, Training Neural Networks with Spark, in the Performance considerations section, we learned how DL4J handles memory when training or running a model. Because it relies on ND4J, it also utilizes off-heap memory and not only heap memory. Being off-heap, it means that it is outside the scope managed by the JVM's Garbage Collection (GC) mechanism (the memory is allocated outside the JVM). At the JVM level, there are only pointers to off-heap memory locations; they can be passed to the C++ code via the Java Native Interface (JNI, https://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/jniTOC.html) for use in ND4J operations.

In DL4J, it is possible to manage memory allocations using two different approaches: ...