Math and Architectures of Deep Learning

5 Probability distributions in machine learning

This chapter covers

• The role of probability distributions in machine learning
• Working with binomial, multinomial, categorical, Bernoulli, beta, and Dirichlet distributions
• The significance of entropy and cross-entropy in machine learning

Life often requires us to estimate the chances of an event occurring or make a decision in the face of uncertainty. Probability and statistics form the common toolbox to use in such circumstances. In machine learning, we take large feature vectors as inputs. As stated earlier, we can view these feature vectors as points in a high-dimensional space. For instance, gray-level images of size 224 × 224 can be viewed as points in a 50, 176-dimensional space, with each ...