Java Deep Learning Cookbook

How to do it...

1. Add the --executor-memory command-line argument while submitting a job to spark-submit to set on-heap memory for the worker node. For example, we could use --executor-memory 4g to allocate 4 GB of memory.
2. Add the --conf command-line argument to set the off-heap memory for the worker node:

--conf "spark.executor.extraJavaOptions=-Dorg.bytedeco.javacpp.maxbytes=8G"

3. Add the --conf command-line argument to set the off-heap memory for the master node. For example, we could use --conf "spark.driver.memoryOverhead=-Dorg.bytedeco.javacpp.maxbytes=8G" to allocate 8 GB of memory.
4. Add the --driver-memory command-line argument to specify the on-heap memory for the master node. For example, we could use --driver-memory 4g to allocate ...