Programming Machine Learning

Chapter 9Designing the Network

Part I of this book was all about the perceptron. Part II is about the perceptron’s big brother, and the most important idea in this book: the neural network. Neural networks are way more powerful than perceptrons. In ​Where Perceptrons Fail​, we learned that perceptrons need simple data that are linearly separable. By contrast, neural networks can deal with gnarly data, like photos of real-world objects.

Even on a simple dataset like MNIST, our perceptron was just scraping by, making almost one mistake every ten characters. With neural networks, we can aim for an order of magnitude better accuracy: in this part of the book, we’d like to build an MNIST classifier that reaches 99% accuracy—one error every 100 characters. ...