Hands-On Mathematics for Deep Learning

Summary

In this chapter, we covered a number of different kinds of optimization, such as convex and non-convex optimization, as well as what makes optimization such a challenging problem. We also had a look at how to define an optimization problem and explored a variety of methods, including population methods, simulated annealing, and gradient descent-based methods. In later chapters, we'll come to understand how optimization is used in deep learning and why it is such an important field for us to understand.

In the next chapter, we will learn about graph theory and its uses in the field to solve various problems.