Chapter 6 Mini-batch Block-coordinate Newton Method
Nowadays, big data challenge is one of the major challenge in machine learning. Stochastic approximation and coordinate descent approaches are very effective to deal with the challenge, as they make each iteration of the learning algorithm independent of number of data points and number of features, respectively. In this chapter, we have combined the best of stochastic approximation and coordinate descent approaches with second order methods to propose mini-bath block-coordinate Newton (MBN) method. We find that MBN does not perform well as per the expectations and lags behind even the pure Newton method in term of training time. This is due to the double sampling ...
Get Stochastic Optimization for Large-scale Machine Learning now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
