Stochastic Optimization for Large-scale Machine Learning

Chapter 5 Learning and Data Access

DOI: 10.1201/9781003240167-5

The training time of models has two major components time to access the data and time to process (learn from) the data. So far, the research has focused only on the second part, i.e., learning from the data. In this chapter, we have proposed one possible solution to handle the big data problems in machine learning. The idea is to reduce the training time through reducing data access time by proposing systematic sampling and cyclic/sequential sampling to select mini-batches from the dataset. To prove the effectiveness of proposed sampling techniques, we have used empirical risk minimization, which is commonly used machine learning problem, for strongly convex and smooth case. The ...