AI for Retail

CHAPTER 7Optimization

Mathematical optimization (or just colloquially optimization) is a cornerstone of modern AI. Without optimization, you cannot have modern AI. Optimization is the field of mathematics that deals with trying to find the minimum or maximum of some objective function subject to some constraints, from some set of available solutions. In layman terms, as the image in Figure 7.1 demonstrates, you are just trying to find the bottom of a bowl. That's it. Sometimes it's really easy, and sometimes it takes a computer the size of Yankee Stadium. It all depends on the problem.

FIGURE 7.1 Gradient descent trajectory over a non‐convex function.

Source: https://towardsdatascience.com/on-optimization-of-deep-neural-networks-21de9e83e1 last accessed 20 Dec 2022 / Neural Information Processing Systems.

Linear Regression

Before 1910, there were no computers. So if you wanted to solve an optimization problem, you had to find a closed‐form/analytical solution, meaning you had to find an equation that gives you the answer directly in one shot. For example, for any of you who studied linear regressions, that has a nice closed‐form analytical solution as well, which is why it is so popular.

If you are not familiar with linear regressions, let's do a quick example. Let's say we had a database of cars and we wanted to predict the MPG (miles per gallon) of a new car assuming ...