Programming Machine Learning

What You Just Learned

In the last few pages, you got acquainted with a new loss function (the cross-entropy loss) and a few new concepts, like numerical stability. The focus of this chapter, however, was on implementing the neural network that we designed in the previous chapter. Here is all the code that we came up with:

10_building/forward_propagation.py

import numpy as np


def sigmoid(z):
    return 1 / (1 + np.exp(-z))


def softmax(logits):
    exponentials = np.exp(logits)
    return exponentials / np.sum(exponentials, axis=1).reshape(-1, 1)


def loss(Y, y_hat):
    return -np.sum(Y * np.log(y_hat)) / Y.shape[0]


def prepend_bias(X):
    return np.insert(X, 0, 1, axis=1)


def forward(X, w1, ...