Statistics for Machine Learning

Deep auto encoders

The auto encoder neural network is an unsupervised learning algorithm that applies backpropagation setting the target values to be equal to the inputs y⁽ⁱ⁾ = x⁽ⁱ⁾. Auto encoder tries to learn a function h_w,b(x) ≈ x, means it tries to learn an approximation to the identity function, so that output    that is similar to x.

Though trying to replicate the identity function seems trivial function to learn, by placing the constraints on the network, such as by limiting number of hidden units, we can discover interesting structures ...