Probabilistic Deep Learning

Probabilistic Deep Learning

by Elvis Murina, Oliver Duerr, Beate Sick
Released November 2020
Publisher(s): Manning Publications
ISBN: 9781617296079

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

Probabilistic Deep Learning: With Python, Keras and TensorFlow Probability teaches the increasingly popular probabilistic approach to deep learning that allows you to refine your results more quickly and accurately without much trial-and-error testing. Emphasizing practical techniques that use the Python-based Tensorflow Probability Framework, you’ll learn to build highly-performant deep learning applications that can reliably handle the noise and uncertainty of real-world data.

About the Technology
The world is a noisy and uncertain place. Probabilistic deep learning models capture that noise and uncertainty, pulling it into real-world scenarios. Crucial for self-driving cars and scientific testing, these techniques help deep learning engineers assess the accuracy of their results, spot errors, and improve their understanding of how algorithms work.

About the Book
Probabilistic Deep Learning is a hands-on guide to the principles that support neural networks. Learn to improve network performance with the right distribution for different data types, and discover Bayesian variants that can state their own uncertainty to increase accuracy. This book provides easy-to-apply code and uses popular frameworks to keep you focused on practical applications.

What's Inside
  • Explore maximum likelihood and the statistical basis of deep learning
  • Discover probabilistic models that can indicate possible outcomes
  • Learn to use normalizing flows for modeling and generating complex distributions
  • Use Bayesian neural networks to access the uncertainty in the model


About the Reader
For experienced machine learning developers.

About the Authors
Oliver Dürr is a professor at the University of Applied Sciences in Konstanz, Germany. Beate Sick holds a chair for applied statistics at ZHAW and works as a researcher and lecturer at the University of Zurich. Elvis Murina is a data scientist.

Quotes
A deep dive through the choppy probabilistic waters that will help reveal the treasures hidden beneath the surface.
- Richard Vaughan, Purple Monkey Collective

Read this book if you are curious about what really happens inside a deep learning network.
- Kim Falk Jorgensen, Binary Vikings

This book opens up a completely new view on many aspects of deep learning.
- Zalán Somogyváry, Vienna University of Technology

A comprehensive, thorough walkthrough in the marvelous world of probabilistic deep learning, with lots of practical examples.
- Diego Casella, Centrica Business Solutions Belgium

Publisher resources

View/Submit Errata

Table of contents

  1. Probabilistic Deep Learning
  2. Copyright
  3. brief contents
  4. contents
  5. front matter
    1. preface
    2. acknowledgments
    3. about this book
      1. Who should read this book
      2. How this book is organized: A roadmap
      3. About the code
      4. liveBook discussion forum
    4. about the authors
    5. about the cover illustration
  6. Part 1. Basics of deep learning
  7. 1 Introduction to probabilistic deep learning
    1. 1.1 A first look at probabilistic models
    2. 1.2 A first brief look at deep learning (DL)
      1. 1.2.1 A success story
    3. 1.3 Classification
      1. 1.3.1 Traditional approach to image classification
      2. 1.3.2 Deep learning approach to image classification
      3. 1.3.3 Non-probabilistic classification
      4. 1.3.4 Probabilistic classification
      5. 1.3.5 Bayesian probabilistic classification
    4. 1.4 Curve fitting
      1. 1.4.1 Non-probabilistic curve fitting
      2. 1.4.2 Probabilistic curve fitting
      3. 1.4.3 Bayesian probabilistic curve fitting
    5. 1.5 When to use and when not to use DL?
      1. 1.5.1 When not to use DL
      2. 1.5.2 When to use DL
      3. 1.5.3 When to use and when not to use probabilistic models?
    6. 1.6 What you’ll learn in this book
    7. Summary
  8. 2 Neural network architectures
    1. 2.1 Fully connected neural networks (fcNNs)
      1. 2.1.1 The biology that inspired the design of artificial NNs
      2. 2.1.2 Getting started with implementing an NN
      3. 2.1.3 Using a fully connected NN (fcNN) to classify images
    2. 2.2 Convolutional NNs for image-like data
      1. 2.2.1 Main ideas in a CNN architecture
      2. 2.2.2 A minimal CNN for edge lovers
      3. 2.2.3 Biological inspiration for a CNN architecture
      4. 2.2.4 Building and understanding a CNN
    3. 2.3 One-dimensional CNNs for ordered data
      1. 2.3.1 Format of time-ordered data
      2. 2.3.2 What’s special about ordered data?
      3. 2.3.3 Architectures for time-ordered data
    4. Summary
  9. 3 Principles of curve fitting
    1. 3.1 “Hello world” in curve fitting
      1. 3.1.1 Fitting a linear regression model based on a loss function
    2. 3.2 Gradient descent method
      1. 3.2.1 Loss with one free model parameter
      2. 3.2.2 Loss with two free model parameters
    3. 3.3 Special DL sauce
      1. 3.3.1 Mini-batch gradient descent
      2. 3.3.2 Using SGD variants to speed up the learning
      3. 3.3.3 Automatic differentiation
    4. 3.4 Backpropagation in DL frameworks
      1. 3.4.1 Static graph frameworks
      2. 3.4.2 Dynamic graph frameworks
    5. Summary
  10. Part 2. Maximum likelihood approaches for probabilistic DL models
  11. 4 Building loss functions with the likelihood approach
    1. 4.1 Introduction to the MaxLike principle: The mother of all loss functions
    2. 4.2 Deriving a loss function for a classification problem
      1. 4.2.1 Binary classification problem
      2. 4.2.2 Classification problems with more than two classes
      3. 4.2.3 Relationship between NLL, cross entropy, and Kullback-Leibler divergence
    3. 4.3 Deriving a loss function for regression problems
      1. 4.3.1 Using a NN without hidden layers and one output neuron for modeling a linear relationship between input and output
      2. 4.3.2 Using a NN with hidden layers to model non-linear relationships between input and output
      3. 4.3.3 Using an NN with additional output for regression tasks with nonconstant variance
    4. Summary
  12. 5 Probabilistic deep learning models with TensorFlow Probability
    1. 5.1 Evaluating and comparing different probabilistic prediction models
    2. 5.2 Introducing TensorFlow Probability (TFP)
    3. 5.3 Modeling continuous data with TFP
      1. 5.3.1 Fitting and evaluating a linear regression model with constant variance
      2. 5.3.2 Fitting and evaluating a linear regression model with a nonconstant standard deviation
    4. 5.4 Modeling count data with TensorFlow Probability
      1. 5.4.1 The Poisson distribution for count data
      2. 5.4.2 Extending the Poisson distribution to a zero-inflated Poisson (ZIP) distribution
    5. Summary
  13. 6 Probabilistic deep learning models in the wild
    1. 6.1 Flexible probability distributions in state-of-the-art DL models
      1. 6.1.1 Multinomial distribution as a flexible distribution
      2. 6.1.2 Making sense of discretized logistic mixture
    2. 6.2 Case study: Bavarian roadkills
    3. 6.3 Go with the flow: Introduction to normalizing flows (NFs)
      1. 6.3.1 The principle idea of NFs
      2. 6.3.2 The change of variable technique for probabilities
      3. 6.3.3 Fitting an NF to data
      4. 6.3.4 Going deeper by chaining flows
      5. 6.3.5 Transformation between higher dimensional spaces*
      6. 6.3.6 Using networks to control flows
      7. 6.3.7 Fun with flows: Sampling faces
    4. Summary
  14. Part 3. Bayesian approaches for probabilistic DL models
  15. 7 Bayesian learning
    1. 7.1 What’s wrong with non-Bayesian DL: The elephant in the room
    2. 7.2 The first encounter with a Bayesian approach
      1. 7.2.1 Bayesian model: The hacker’s way
      2. 7.2.2 What did we just do?
    3. 7.3 The Bayesian approach for probabilistic models
      1. 7.3.1 Training and prediction with a Bayesian model
      2. 7.3.2 A coin toss as a Hello World example for Bayesian models
      3. 7.3.3 Revisiting the Bayesian linear regression model
    4. Summary
  16. 8 Bayesian neural networks
    1. 8.1 Bayesian neural networks (BNNs)
    2. 8.2 Variational inference (VI) as an approximative Bayes approach
      1. 8.2.1 Looking under the hood of VI*
      2. 8.2.2 Applying VI to the toy problem*
    3. 8.3 Variational inference with TensorFlow Probability
    4. 8.4 MC dropout as an approximate Bayes approach
      1. 8.4.1 Classical dropout used during training
      2. 8.4.2 MC dropout used during train and test times
    5. 8.5 Case studies
      1. 8.5.1 Regression case study on extrapolation
      2. 8.5.2 Classification case study with novel classes
    6. Summary
  17. Glossary of terms and abbreviations
  18. index

Product information

  • Title: Probabilistic Deep Learning
  • Author(s): Elvis Murina, Oliver Duerr, Beate Sick
  • Release date: November 2020
  • Publisher(s): Manning Publications
  • ISBN: 9781617296079