book

Bayesian Optimization in Action

Name: Bayesian Optimization in Action
Author: Quan Nguyen
ISBN: 9781633439078

by Quan Nguyen

December 2023

Intermediate to advanced

424 pages

12h 28m

English

Manning Publications

Audiobook available

Read now

Unlock full access

Inside front cover
Bayesian Optimization in Action
Copyright
dedication
contents
front matter
forewordsprefaceacknowledgmentsabout this bookWho should read this book?How this book is organized: A roadmapAbout the codeliveBook discussion forumabout the authorAbout the technical editorabout the cover illustration
1 Introduction to Bayesian optimization
1.1 Finding the optimum of an expensive black box function1.1.1 Hyperparameter tuning as an example of an expensive black box optimization problem1.1.2 The problem of expensive black box optimization1.1.3 Other real-world examples of expensive black box optimization problems1.2 Introducing Bayesian optimization1.2.1 Modeling with a Gaussian process1.2.2 Making decisions with a BayesOpt policy1.2.3 Combining the GP and the optimization policy to form the optimization loop1.2.4 BayesOpt in action1.3 What will you learn in this book?Summary
Part 1. Modeling with Gaussian processes
2 Gaussian processes as distributions over functions
2.1 How to sell your house the Bayesian way2.2 Modeling correlations with multivariate Gaussian distributions and Bayesian updates2.2.1 Using multivariate Gaussian distributions to jointly model multiple variables2.2.2 Updating MVN distributions2.2.3 Modeling many variables with high-dimensional Gaussian distributions2.3 Going from a finite to an infinite Gaussian2.4 Implementing GPs in Python2.4.1 Setting up the training data2.4.2 Implementing a GP class2.4.3 Making predictions with a GP2.4.4 Visualizing predictions of a GP2.4.5 Going beyond one-dimensional objective functions2.5 ExerciseSummary
3 Customizing a Gaussian process with the mean and covariance functions
3.1 The importance of priors in Bayesian models3.2 Incorporating what you already know into a GP3.3 Defining the functional behavior with the mean function3.3.1 Using the zero mean function as the base strategy3.3.2 Using the constant function with gradient descent3.3.3 Using the linear function with gradient descent3.3.4 Using the quadratic function by implementing a custom mean function3.4 Defining variability and smoothness with the covariance function3.4.1 Setting the scales of the covariance function3.4.2 Controlling smoothness with different covariance functions3.4.3 Modeling different levels of variability with multiple length scales3.5 ExerciseSummary

Part 2. Making decisions with Bayesian optimization
4 Refining the best result with improvement-based policies
4.1 Navigating the search space in BayesOpt4.1.1 The BayesOpt loop and policies4.1.2 Balancing exploration and exploitation4.2 Finding improvement in BayesOpt4.2.1 Measuring improvement with a GP4.2.2 Computing the Probability of Improvement4.2.3 Running the PoI policy4.3 Optimizing the expected value of improvement4.4 Exercises4.4.1 Exercise 1: Encouraging exploration with PoI4.4.2 Exercise 2: BayesOpt for hyperparameter tuningSummary
5 Exploring the search space with bandit-style policies
5.1 Introduction to the MAB problem5.1.1 Finding the best slot machine at a casino5.1.2 From MAB to BayesOpt5.2 Being optimistic under uncertainty with the Upper Confidence Bound policy5.2.1 Optimism under uncertainty5.2.2 Balancing exploration and exploitation5.2.3 Implementation with BoTorch5.3 Smart sampling with the Thompson sampling policy5.3.1 One sample to represent the unknown5.3.2 Implementation with BoTorch5.4 Exercises5.4.1 Exercise 1: Setting an exploration schedule for the UCB5.4.2 Exercise 2: BayesOpt for hyperparameter tuningSummary
6 Using information theory with entropy-based policies
6.1 Measuring knowledge with information theory6.1.1 Measuring uncertainty with entropy6.1.2 Looking for a remote control using entropy6.1.3 Binary search using entropy6.2 Entropy search in BayesOpt6.2.1 Searching for the optimum using information theory6.2.2 Implementing entropy search with BoTorch6.3 Exercises6.3.1 Exercise 1: Incorporating prior knowledge into entropy search6.3.2 Exercise 2: Bayesian optimization for hyperparameter tuningSummary
Part 3. Extending Bayesian optimization to specialized settings
7 Maximizing throughput with batch optimization
7.1 Making multiple function evaluations simultaneously7.1.1 Making use of all available resources in parallel7.1.2 Why can’t we use regular BayesOpt policies in the batch setting?7.2 Computing the improvement and upper confidence bound of a batch of points7.2.1 Extending optimization heuristics to the batch setting7.2.2 Implementing batch improvement and UCB policies7.3 Exercise 1: Extending TS to the batch setting via resampling7.4 Computing the value of a batch of points using information theory7.4.1 Finding the most informative batch of points with cyclic refinement7.4.2 Implementing batch entropy search with BoTorch7.5 Exercise 2: Optimizing airplane designsSummary
8 Satisfying extra constraints with constrained optimization
8.1 Accounting for constraints in a constrained optimization problem8.1.1 Constraints can change the solution of an optimization problem8.1.2 The constraint-aware BayesOpt framework8.2 Constraint-aware decision-making in BayesOpt8.3 Exercise 1: Manual computation of constrained EI8.4 Implementing constrained EI with BoTorch8.5 Exercise 2: Constrained optimization of airplane designSummary
9 Balancing utility and cost with multifidelity optimization
9.1 Using low-fidelity approximations to study expensive phenomena9.2 Multifidelity modeling with GPs9.2.1 Formatting a multifidelity dataset9.2.2 Training a multifidelity GP9.3 Balancing information and cost in multifidelity optimization9.3.1 Modeling the costs of querying different fidelities9.3.2 Optimizing the amount of information per dollar to guide optimization9.4 Measuring performance in multifidelity optimization9.5 Exercise 1: Visualizing average performance in multifidelity optimization9.6 Exercise 2: Multifidelity optimization with multiple low-fidelity approximationsSummary
10 Learning from pairwise comparisons with preference optimization
10.1 Black-box optimization with pairwise comparisons10.2 Formulating a preference optimization problem and formatting pairwise comparison data10.3 Training a preference-based GP10.4 Preference optimization by playing king of the hillSummary
11 Optimizing multiple objectives at the same time
11.1 Balancing multiple optimization objectives with BayesOpt11.2 Finding the boundary of the most optimal data points11.3 Seeking to improve the optimal data boundary11.4 Exercise: Multiobjective optimization of airplane designSummary
Part 4. Special Gaussian process models
12 Scaling Gaussian processes to large datasets
12.1 Training a GP on a large dataset12.1.1 Setting up the learning task12.1.2 Training a regular GP12.1.3 Problems with training a regular GP12.2 Automatically choosing representative points from a large dataset12.2.1 Minimizing the difference between two GPs12.2.2 Training the model in small batches12.2.3 Implementing the approximate model12.3 Optimizing better by accounting for the geometry of the loss surface12.4 ExerciseSummary
13 Combining Gaussian processes with neural networks
13.1 Data that contains structures13.2 Capturing similarity within structured data13.2.1 Using a kernel with GPyTorch13.2.2 Working with images in PyTorch13.2.3 Computing the covariance of two images13.2.4 Training a GP on image data13.3 Using neural networks to process complex structured data13.3.1 Why use neural networks for modeling?13.3.2 Implementing the combined model in GPyTorchSummary
Appendix. Solutions to the exercises
A.1 Chapter 2: Gaussian processes as distributions over functionsA.2 Chapter 3: Incorporating prior knowledge with the mean and covariance functionsA.3 Chapter 4: Refining the best result with improvement-based policiesA.3.1 Exercise 1: Encouraging exploration with Probability of ImprovementA.3.2 Exercise 2: BayesOpt for hyperparameter tuningA.4 Chapter 5: Exploring the search space with bandit-style policiesA.4.1 Exercise 1: Setting an exploration schedule for Upper Confidence BoundA.4.2 Exercise 2: BayesOpt for hyperparameter tuningA.5 Chapter 6: Using information theory with entropy-based policiesA.5.1 Exercise 1: Incorporating prior knowledge into entropy searchA.5.2 Exercise 2: BayesOpt for hyperparameter tuningA.6 Chapter 7: Maximizing throughput with batch optimizationA.6.1 Exercise 1: Extending TS to the batch setting via resamplingA.6.2 Exercise 2: Optimizing airplane designsA.7 Chapter 8: Satisfying extra constraints with constrained optimizationA.7.1 Exercise 1: Manual computation of constrained EIA.7.2 Exercise 2: Constrained optimization of airplane designA.8 Chapter 9: Balancing utility and cost with multifidelity optimizationA.8.1 Exercise 1: Visualizing average performance in multifidelity optimizationA.8.2 Exercise 2: Multifidelity optimization with multiple low-fidelity approximationsA.9 Chapter 11: Optimizing multiple objectives at the same timeA.10 Chapter 12: Scaling Gaussian processes to large data sets
index
Inside back cover

Content preview from Bayesian Optimization in Action

6 Using information theory with entropy-based policies

This chapter covers

Entropy as an information-theoretic measure of uncertainty
Information gain as a method of reducing entropy
BayesOpt policies that use information theory for their search

We saw in chapter 4 that by aiming to improve from the best value achieved so far, we can design improvement-based BayesOpt policies, such as Probability of Improvement (POI) and Expected Improvement (EI). In chapter 5, we used multi-armed bandit (MAB) policies to obtain Upper Confidence Bound (UCB) and Thompson sampling (TS), each of which uses a unique heuristic to balance exploration and exploitation in the search for the global optimum of the objective function.

In this chapter, we learn about another ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Bayesian Optimization: Theory and Practice Using Python

Publisher Resources

ISBN: 9781633439078Publisher Support Publisher Website

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Bayesian Optimization in Action

by Quan Nguyen

6 Using information theory with entropy-based policies

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.