Java Deep Learning Cookbook

There's more...

Memory configurations are often applied to master/worker nodes separately. Therefore, memory configuration on worker nodes alone may not bring the required results. The approach we take can vary, depending on the cluster resource manager we use. Therefore, it is important to refer to the respective documentation on the different approaches for a specific cluster resource manager. Also, note that the default memory settings in the cluster resource managers are not appropriate (too low) for libraries (ND4J/DL4J) that heavily rely on off-heap memory space. spark-submit can load the configurations in two different ways. One way is to use the command line, as we discussed previously, while another one is to specify the configuration ...