Hyperparameter Optimization in Machine Learning: Make Your Machine Learning and Deep Learning Models More Efficient

© Tanmay Agrawal 2021

T. AgrawalHyperparameter Optimization in Machine Learninghttps://doi.org/10.1007/978-1-4842-6579-6_3

3. Solving Time and Memory Constraints

Tanay Agrawal¹  

(1)

Bangalore, Karnataka, India

 

We face two major problems while tuning hyperparameters:

• Memory constraint : Sometimes we have to deal with hundreds of gigabytes of data. We cannot store such a huge amount of data in RAM. While training a neural network, we send data in batches. One of the possible solutions is larger memory, which is not feasible always.

• Time/computation constraint : Let’s say our data fits into memory, but we are training a deep neural network (DNN) or there is a huge search space for hyperparameter optimization. This can consume a great amount of time.