Hands-On Mathematics for Deep Learning

Understanding optimization and it's different types

In optimization, our goal is to either minimize or maximize a function. For example, a business wants to minimize its costs while maximizing its profits or a shopper might want to get as much as possible while spending as little as possible. Therefore, the goal of optimization is to find the best case of , which is denoted by x^* (where x is a set of points), that satisfies certain criteria. These criteria are, for our purposes, mathematical functions known as objective functions.

For example, let's suppose we have the equation. If we plot it, we get the following graph:

You will recall from ...