Applied Deep Learning with TensorFlow 2: Learn to Implement Advanced Deep Learning Techniques with Python

© Umberto Michelucci 2022

U. MichelucciApplied Deep Learning with TensorFlow 2https://doi.org/10.1007/978-1-4842-8020-1_1

1. Optimization and Neural Networks

Umberto Michelucci¹  

(1)

Dübendorf, Switzerland

 

This chapter contains a basic introduction to the most important concepts of optimization and explains how they are related to neural networks. This chapter doesn’t go into detail, leaving longer discussions to the following chapters. But at the end of this chapter, you should have a basic understanding of the most important concepts and challenges related to neural networks. This chapter covers the problem of learning, constrained and unconstrained optimization problems, what optimization algorithms are, and the gradient descent algorithm and ...