Deep Learning Essentials

Stacked/continuous RBM

A deep-belief network (DBN) is simply a few RBMs stacked on top of one another. The output from the previous RBM becomes the input of the following RBM. In 2006, Hinton proposed a fast, greedy algorithm in his paper: A fast learning algorithm for deep belief nets, that can learn deep, directed belief networks one layer at a time. DBN learns a hierarchical representation of input and aims to reconstruct the data, therefore it is very useful, especially in an unsupervised setting.

For continuous input, one can refer to another model called continuous restricted Boltzmann machines, which utilize a different type of contrastive divergence sampling. Such models can deal with image pixels or word vectors that are normalized ...