R Deep Learning Cookbook

Fine-tuning the parameters of the autoencoder

The autoencoder involves a couple of parameters to tune, depending on the type of autoencoder we are working on. The major parameters in an autoencoder include the following:

• Number of nodes in any hidden layer
• Number of hidden layers applicable for deep autoencoders
• Activation unit such as sigmoid, tanh, softmax, and ReLU activation functions
• Regularization parameters or weight decay terms on hidden unit weights
• Fraction of the signal to be corrupted in a denoising autoencoder
• Sparsity parameters in sparse autoencoders that control the expected activation of neurons in hidden layers
• Batch size, if using batch gradient descent learning; learning rate and momentum parameter for stochastic gradient ...