Java Deep Learning Cookbook

There's more...

The ReLU activation function is non-linear, hence, the backpropagation of errors can easily be performed. Backpropagation is the backbone of neural networks. This is the learning algorithm that computes gradient descent with respect to weights across neurons. The following are ReLU variations currently supported in DL4J:

• ReLU: The standard ReLU activation function:

public static final Activation RELU

• ReLU6: ReLU activation, which is capped at 6, where 6 is an arbitrary choice:

public static final Activation RELU6

• RReLU: The randomized ReLU activation function:

public static final Activation RRELU

• ThresholdedReLU: Threshold ReLU:

public static final Activation THRESHOLDEDRELU

There are a few more implementations, such ...