Bayesian Optimization in Action

4 Refining the best result with improvement-based policies

This chapter covers

• The BayesOpt loop
• The tradeoff between exploitation and exploration in a BayesOpt policy
• Improvement as a criterion for finding new data points
• BayesOpt policies that use improvement

In this chapter, we first remind ourselves of the iterative nature of BayesOpt: we alternate between training a Gaussian process (GP) on the collected data and finding the next data point to label using a BayesOpt policy. This forms a virtuous cycle in which our past data inform future decisions. We then talk about what we look for in a BayesOpt policy: a decision-making algorithm that decides which data point to label. A good BayesOpt policy needs to balance sufficiently exploring the ...