Stochastic Optimization for Large-scale Machine Learning

Chapter 2 Optimization Problem, Solvers, Challenges and Research Directions

DOI: 10.1201/9781003240167-2

Behind the scenes, machine learning problems are actually optimization problems. So, optimization problem formulation and selection of appropriate solvers are very important for the final performance of machine learning models. We take the example of SVM to illustrate that how a problem can be formulated in multiple ways and how each formulation can be solved using multiple solvers. SVM is a binary linear classifier which has been extended to non-linear data using Kernels and multi-class data using various techniques like one-versus-one, one-versus-rest and Crammer Singer SVM, etc. SVM with a linear Kernel is called linear SVM and one with ...