Programming Machine Learning

Adding More Tricks to Your Bag

Picking the right activation functions is a crucial decision, but when you design a neural network, you face plenty more. You decide how to initialize the weights, which GD algorithm to use, what kind of regularization to apply, and so forth. You have a wide range of techniques to choose from, and new ones come up all the time.

It would be pointless to go into too much detail about all the popular techniques available today. You could fill entire volumes with them. Besides, some of them might be old-fashioned and quaint by the time you read this book.

For those reasons, this section doesn’t aspire to be comprehensive. See it like a quick shopping spree in the mall of modern neural networks: we’ll look at a handful ...