Essential Math for AI

Chapter 8. Probabilistic Generative Models

AI ties up all the math that I know together, and I have been getting to know math for years.

H.

If machines are ever to be endowed with an understanding of the world around them, and an ability to recreate it, like we do when we imagine, dream, draw, create songs, watch movies, or write books, then generative models are one significant step in that direction. We need to get these models right if we are ever going to achieve general artificial intelligence.

Generative models are built on the assumption that we can only interpret input data correctly if our model has learned the underlying statistical structure of this data. This is loosely analogous to our dreaming process, which points to the possibility that our brain has learned a model that is able to virtually recreate our environment.

In this chapter, we still have the mathematical structure of training function, loss function, and optimization presented throughout the book. However, unlike in the first few chapters, we aim to learn probability distributions, instead of deterministic functions. The overarching theme is that there is training data, and we want to come up with a mathematical model that generates new data similar to it.

There are two quantities of interest:

• The true (and unknown) joint probability distribution of the features of the input data $p_{data}\left(\overrightarrow{x}\right)$.

• The model joint probability distribution of the features of the data along with the parameters ...