Scala Machine Learning Projects

Activation functions

In most cases, you can use the ReLU activation function in the hidden layers. It is a bit faster to compute than other activation functions, and gradient descent does not get stuck as much on plateaus compared to the logistic function or the hyperbolic tangent function, which usually saturated at one.

For the output layer, the softmax activation function is generally a good choice for classification tasks. For regression tasks, you can simply use no activation function. Other activation functions include Sigmoid and Tanh. The current implementation of the H2O-based deep learning model supports the following activation functions:

• ExpRectifier
• ExpRectifierWithDropout
• Maxout
• MaxoutWithDropout
• Rectifier
• RectifierWthDropout ...