Skip to Main Content
Grokking Deep Learning
book

Grokking Deep Learning

by Andrew W. Trask
February 2019
Intermediate to advanced content levelIntermediate to advanced
336 pages
9h 29m
English
Manning Publications
Content preview from Grokking Deep Learning

Chapter 9. Modeling probabilities and nonlinearities: activation functions

In this chapter

What is an activation function?

Standard hidden activation functions

  • Sigmoid
  • Tanh

Standard output activation functions

  • Softmax

Activation function installation instructions

“I know that 2 and 2 make 4—& should be glad to prove it too if I could—though I must say if by any sort of process I could convert 2 & 2 into five it would give me much greater pleasure.”

George Gordon Byron, letter to Annabella Milbanke, November 10, 1813

What is an activation function?

It’s a function applied to the neurons in a layer during prediction

An activation function is a function applied to the neurons in a layer during prediction. This should seem very familiar, ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Start your free trial

You might also like

Generative Deep Learning

Generative Deep Learning

David Foster
Deep Learning with PyTorch

Deep Learning with PyTorch

Eli Stevens, Thomas Viehmann, Luca Pietro Giovanni Antiga

Publisher Resources

ISBN: 9781617293702Supplemental ContentPublisher SupportOtherPublisher WebsiteSupplemental ContentPurchase Link