Bayesian Optimization in Action, Video Edition

Bayesian Optimization in Action, Video Edition

by Quan Nguyen
Released December 2023
Publisher(s): Manning Publications
ISBN: None

Watch it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Video description

In Video Editions the narrator reads the book while the content, figures, code listings, diagrams, and text appear on the screen. Like an audiobook that you can also watch as a video.

Bayesian optimization helps pinpoint the best configuration for your machine learning models with speed and accuracy. Put its advanced techniques into practice with this hands-on guide.

In Bayesian Optimization in Action you will learn how to:

  • Train Gaussian processes on both sparse and large data sets
  • Combine Gaussian processes with deep neural networks to make them flexible and expressive
  • Find the most successful strategies for hyperparameter tuning
  • Navigate a search space and identify high-performing regions
  • Apply Bayesian optimization to cost-constrained, multi-objective, and preference optimization
  • Implement Bayesian optimization with PyTorch, GPyTorch, and BoTorch

Bayesian Optimization in Action shows you how to optimize hyperparameter tuning, A/B testing, and other aspects of the machine learning process by applying cutting-edge Bayesian techniques. Using clear language, illustrations, and concrete examples, this book proves that Bayesian optimization doesn’t have to be difficult! You’ll get in-depth insights into how Bayesian optimization works and learn how to implement it with cutting-edge Python libraries. The book’s easy-to-reuse code samples let you hit the ground running by plugging them straight into your own projects.

About the Technology
In machine learning, optimization is about achieving the best predictions—shortest delivery routes, perfect price points, most accurate recommendations—in the fewest number of steps. Bayesian optimization uses the mathematics of probability to fine-tune ML functions, algorithms, and hyperparameters efficiently when traditional methods are too slow or expensive.

About the Book
Bayesian Optimization in Action teaches you how to create efficient machine learning processes using a Bayesian approach. In it, you’ll explore practical techniques for training large datasets, hyperparameter tuning, and navigating complex search spaces. This interesting book includes engaging illustrations and fun examples like perfecting coffee sweetness, predicting weather, and even debunking psychic claims. You’ll learn how to navigate multi-objective scenarios, account for decision costs, and tackle pairwise comparisons.

What's Inside
  • Gaussian processes for sparse and large datasets
  • Strategies for hyperparameter tuning
  • Identify high-performing regions
  • Examples in PyTorch, GPyTorch, and BoTorch


About the Reader
For machine learning practitioners who are confident in math and statistics.

About the Author
Quan Nguyen is a research assistant at Washington University in St. Louis. He writes for the Python Software Foundation and has authored several books on Python programming.

Quotes
Using a hands-on approach, clear diagrams, and real-world examples, Quan lifts the veil off the complexities of Bayesian optimization.
- From the Foreword by Luis Serrano, Author of Grokking Machine Learning

This book teaches Bayesian optimization, starting from its most basic components. You’ll find enough depth to make you comfortable with the tools and methods and enough code to do real work very quickly.
- From the Foreword by David Sweet, Author of Experimentation for Engineers

Combines modern computational frameworks with visualizations and infographics you won’t find anywhere else. It gives readers the confidence to apply Bayesian optimization to real world problems!
- Ravin Kumar, Google

Table of contents

  1. Chapter 1. Introduction to Bayesian optimization
  2. Chapter 1. Introducing Bayesian optimization
  3. Chapter 1. What will you learn in this book?
  4. Chapter 1. Summary
  5. Part 1. Modeling with Gaussian processes
  6. Chapter 2. Gaussian processes as distributions over functions
  7. Chapter 2. Modeling correlations with multivariate Gaussian distributions and Bayesian updates
  8. Chapter 2. Going from a finite to an infinite Gaussian
  9. Chapter 2. Implementing GPs in Python
  10. Chapter 2. Exercise
  11. Chapter 2. Summary
  12. Chapter 3. Customizing a Gaussian process with the mean and covariance functions
  13. Chapter 3. Incorporating what you already know into a GP
  14. Chapter 3. Defining the functional behavior with the mean function
  15. Chapter 3. Defining variability and smoothness with the covariance function
  16. Chapter 3. Exercise
  17. Chapter 3. Summary
  18. Part 2. Making decisions with Bayesian optimization
  19. Chapter 4. Refining the best result with improvement-based policies
  20. Chapter 4. Finding improvement in BayesOpt
  21. Chapter 4. Optimizing the expected value of improvement
  22. Chapter 4. Exercises
  23. Chapter 4. Summary
  24. Chapter 5. Exploring the search space with bandit-style policies
  25. Chapter 5. Being optimistic under uncertainty with the Upper Confidence Bound policy
  26. Chapter 5. Smart sampling with the Thompson sampling policy
  27. Chapter 5. Exercises
  28. Chapter 5. Summary
  29. Chapter 6. Using information theory with entropy-based policies
  30. Chapter 6. Entropy search in BayesOpt
  31. Chapter 6. Exercises
  32. Chapter 6. Summary
  33. Part 3. Extending Bayesian optimization to specialized settings
  34. Chapter 7. Maximizing throughput with batch optimization
  35. Chapter 7. Computing the improvement and upper confidence bound of a batch of points
  36. Chapter 7. Exercise 1: Extending TS to the batch setting via resampling
  37. Chapter 7. Computing the value of a batch of points using information theory
  38. Chapter 7. Exercise 2: Optimizing airplane designs
  39. Chapter 7. Summary
  40. Chapter 8. Satisfying extra constraints with constrained optimization
  41. Chapter 8. Constraint-aware decision-making in BayesOpt
  42. Chapter 8. Exercise 1: Manual computation of constrained EI
  43. Chapter 8. Implementing constrained EI with BoTorch
  44. Chapter 8. Exercise 2: Constrained optimization of airplane design
  45. Chapter 8. Summary
  46. Chapter 9. Balancing utility and cost with multifidelity optimization
  47. Chapter 9. Multifidelity modeling with GPs
  48. Chapter 9. Balancing information and cost in multifidelity optimization
  49. Chapter 9. Measuring performance in multifidelity optimization
  50. Chapter 9. Exercise 1: Visualizing average performance in multifidelity optimization
  51. Chapter 9. Exercise 2: Multifidelity optimization with multiple low-fidelity approximations
  52. Chapter 9. Summary
  53. Chapter 10. Learning from pairwise comparisons with preference optimization
  54. Chapter 10. Formulating a preference optimization problem and formatting pairwise comparison data
  55. Chapter 10. Training a preference-based GP
  56. Chapter 10. Preference optimization by playing king of the hill
  57. Chapter 10. Summary
  58. Chapter 11. Optimizing multiple objectives at the same time
  59. Chapter 11. Finding the boundary of the most optimal data points
  60. Chapter 11. Seeking to improve the optimal data boundary
  61. Chapter 11. Exercise: Multiobjective optimization of airplane design
  62. Chapter 11. Summary
  63. Part 4. Special Gaussian process models
  64. Chapter 12. Scaling Gaussian processes to large datasets
  65. Chapter 12. Automatically choosing representative points from a large dataset
  66. Chapter 12. Optimizing better by accounting for the geometry of the loss surface
  67. Chapter 12. Exercise
  68. Chapter 12. Summary
  69. Chapter 13. Combining Gaussian processes with neural networks
  70. Chapter 13. Capturing similarity within structured data
  71. Chapter 13. Using neural networks to process complex structured data
  72. Chapter 13. Summary

Product information

  • Title: Bayesian Optimization in Action, Video Edition
  • Author(s): Quan Nguyen
  • Release date: December 2023
  • Publisher(s): Manning Publications
  • ISBN: None