We saw in Chapter 2 how to construct and train (supervised) a traditional multiple-layer feedforward network. In Chapter 3 we learned how to construct and train without supervision a single-layer Restricted Boltzmann Machine (RBM). Now we will explore how to stack multiple RBMs into a deep belief network, train this multiple-layer network without supervision, and append an output layer that is trained with supervision. The resulting network can be enormously powerful.
The algorithm for stacking RBMs is surprisingly simple. Roughly stated, we train the bottom-most ...
