Hands-On Deep Learning Algorithms with Python

The Exponential linear unit function

Exponential linear unit (ELU), like Leaky ReLU, has a small slope for negative values. But instead of having a straight line, it has a log curve, as shown in the following diagram:

It can be expressed as follows:

The ELU function is implemented in Python as follows:

def ELU(x,alpha=0.01):    if x<0:        return ((alpha*(np.exp(x)-1))    else:        return x