Programming Machine Learning

What You Just Learned

In this chapter we designed an artificial neural network—or just a neural network, for short—by joining two perceptrons. Because the output of the first perceptron is also the input of the second, the resulting neural network has three layers: input, hidden, and output. Each layer is made up of a bunch of nodes, or neurons.

In between each pair of consecutive layers, there are two operations: a weighted sum of the first layer, and an activation function. In our design, we used a sigmoid as the activation function for the input layer, and a softmax for the hidden layer. The input and hidden layer also have one bias node each, with a fixed value of 1.

Armed with that knowledge, we designed a network to recognize MNIST characters. ...