Bayesian Optimization in Action

7 Maximizing throughput with batch optimization

This chapter covers

• Making function evaluations in batches
• Extending BayesOpt to the batch setting
• Optimizing hard-to-compute acquisition scores

The BayesOpt loop we have been working with thus far takes in one query at a time and returns the function evaluation for the query before the next query is made. We use this loop in settings where function evaluations can only be made sequentially. However, many real-world scenarios of black box optimization allow the user to evaluate the objective functions in batches. For example, when tuning the hyperparameters of an ML model, we can try out different hyperparameter combinations in parallel if we have access to multiple processing units or computers, ...