Applied Deep Learning: A Case-Based Approach to Understanding Deep Neural Networks

© Umberto Michelucci 2018

Umberto MichelucciApplied Deep Learninghttps://doi.org/10.1007/978-1-4842-3790-8_4

4. Training Neural Networks

Umberto Michelucci¹ 

(1)

toelt.ai, Dübendorf, Switzerland

 

Building complex networks with TensorFlow is quite easy, as you have probably realized by now. A few lines of code are enough to construct networks with thousands (and even more) parameters. It should be clear by now that problems arise while training such networks. It is difficult, unstable, and slow to test hyperparameters, because a run over a few hundred epochs may take hours. This is not only a performance problem; otherwise, it would suffice to use faster and faster hardware. The problem is that very often, the convergence process (the learning) does ...