Hands-On Unsupervised Learning with Python

Deep belief networks

A DBN is a stacked model based on RBMs. The generic structure is shown in the following diagram:

Structure of a generic DBN

The first layer contains visible units, while all of the remaining ones are latent. In an unsupervised scenario, the goal is to learn an unknown distribution, finding out the internal representation of the samples. In fact, when the number of latent units is smaller than the input ones, the model learns how to encode the distribution by using lower-dimensional subspace. Hinton and Osindero (in Hinton, G. E., and Osindero, S., A Fast Learning Algorithm for Deep Belief Nets, Teh Y. W., Neural Computation, ...