Building Machine Learning and Deep Learning Models on Google Cloud Platform: A Comprehensive Guide for Beginners

© Ekaba Bisong 2019

E. . BisongBuilding Machine Learning and Deep Learning Models on Google Cloud Platformhttps://doi.org/10.1007/978-1-4842-4470-8_32

32. Other Considerations for Training the Network

Ekaba Bisong¹ 

(1)

OTTAWA, ON, Canada

 

In this chapter, we will cover some other important techniques to consider when training a deep neural network.

Weight Initialization

Weight initialization is a technique for assigning initial values to the weights (parameters) of the neural network before training (see Figure 32-1). Proper weight initialization may mitigate the effects of vanishing and exploding gradients when training the network. It may also speed up the training process. Two commonly used methods for weight initializations are the Xavier and the ...