Statistics for Machine Learning

Solving methodology

Backpropagation is used to solve deep layers by calculating the error of the network at output units and propagate back through layers to update the weights to reduce error terms.

Thumb rules in designing deep neural networks: Though there is no hard and fast rule for designing neural networks, the following rules will provide some guidelines:

• All hidden layers should have the same number of neurons per layer
• Typically, two hidden layers are good enough to solve the majority of problems
• Using scaling/batch normalization (mean 0, variance 1) for all input variables after each layer improves convergence effectiveness
• Reduction in step size after each iteration improves convergence, in addition to the use of momentum and ...