Math and Architectures of Deep Learning

6 Bayesian tools for machine learning

This chapter covers

• Unsupervised machine learning models
• Bayes’ theorem, conditional probability, entropy, cross-entropy, and conditional entropy
• Maximum likelihood estimation (MLE) and maximum a posteriori (MAP) estimation of model parameters
• Evidence maximization
• KLD
• Gaussian mixture models (GMM) and MLE estimation of GMM parameters

The Bayesian approach to statistics tries to model the world by modeling the uncertainties and prevailing beliefs and knowledge about the system. This is in contrast to the frequentist paradigm, where probability is strictly measured by observing a phenomenon repeatedly and measuring the fraction of time an event occurs. Machine learning, in particular unsupervised machine ...