A Handbook of Mathematical Models with Python

10

Optimization Techniques for Machine Learning

We discussed mathematical optimization techniques in the previous chapter and their necessity in business problems that require minimizing the cost (error) function and in predictive modeling, wherein the machine learns from historical data to predict the future. In Machine Learning (ML), the cost is a loss function or an energy function that is minimized. It can be challenging in most cases to know which optimization algorithm should be considered for a given ML model. Optimization is an iterative process to maximize or minimize an objective function and there is always a trade-off between the number of iteration steps taken and the computational hardship to get to the next step. In this chapter, ...