Java Deep Learning Cookbook

There's more...

Learning rate is one of the factors that decides the efficiency of the neural network. A high learning rate will diverge from the actual output, while a low learning rate will result in slow learning due to slow convergence. Neural network efficiency also depends on the weights that we assign to the neurons in every layer. Hence, a uniform distribution of weights during the early stages of training might help.

The most commonly followed approach is to introduce dropouts to the layers. This forces the neural network to ignore some of the neurons during the training process. This will effectively prevent the neural network from memorizing the prediction process. How do we find out if a network has memorized the results? Well, ...