Chapter 9. Modeling probabilities and nonlinearities: activation functions
In this chapter
What is an activation function?
Standard hidden activation functions
- Sigmoid
- Tanh
Standard output activation functions
- Softmax
Activation function installation instructions
“I know that 2 and 2 make 4—& should be glad to prove it too if I could—though I must say if by any sort of process I could convert 2 & 2 into five it would give me much greater pleasure.”
George Gordon Byron, letter to Annabella Milbanke, November 10, 1813
What is an activation function?
It’s a function applied to the neurons in a layer during prediction
An activation function is a function applied to the neurons in a layer during prediction. This should seem very familiar, ...
Get Grokking Deep Learning now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
