Thoughtful Machine Learning with Python

Chapter 8. Neural Networks

Humans are amazing pattern matchers. When we come out of the womb, we are able to make sense of the surrounding chaos until we have learned how to operate effectively in the world. This of course has to do with our upbringing, our environment, but most importantly our brain.

Your brain contains roughly 86 billion neurons that talk to one another over a network of synapses. These neurons are able to control your body functions as well as form thoughts, memories, and mental models. Each one of these neurons acts as part of a computer network, taking inputs and sending outputs to other neurons, all communicating in orchestrated fashion.

Mathematicians decided a long time ago it would be interesting to try to piece together mathematical representations of our brains, called neural networks. While the original research is over 60 years old, many of the techniques conceived back then still apply today and can be used to build models to tricky to compute problems.

In this chapter we will discuss neural networks in depth. We’ll cover:

• Threshold logic, or how to build a Boolean function

• Neural networks as chaotic Boolean functions

• How to construct a feed-forward neural net

• Testing strategies for neural networks through gradient descent

• An example of classifying the language of handwritten text

What Is a Neural Network?

In a lot of ways neural networks are the perfect machine learning construct. They are a way of mapping inputs to a general output ...