Deep Learning

Chapter 2. Foundations of Neural Networks and Deep Learning

With your feet in the air and your head on the ground Try this trick and spin it, yeah Your head will collapse But there’s nothing in it And you’ll ask yourself

Where is my mind

The Pixies, “Where is My Mind?”

Neural Networks

Neural networks are a computational model that shares some properties with the animal brain in which many simple units are working in parallel with no centralized control unit. The weights between the units are the primary means of long-term information storage in neural networks. Updating the weights is the primary way the neural network learns new information.

In Chapter 1 we discussed modeling sets of equations in the form of the equation Ax = b. In the context of neural networks, the A matrix is still the input data and the b column vector is still the labels or outcomes for each row in the A matrix. The weights on the neural network connections becomes x (the parameter vector).

The behavior of neural networks is shaped by its network architecture. A network’s architecture can be defined (in part) by the following:

• Number of neurons
• Number of layers
• Types of connections between layers

The most well-known and simplest-to-understand neural network is the feed-forward multilayer neural network. It has an input layer, one or many hidden layers, and a single output layer. Each layer can have a different number of neurons and each layer is fully connected to the adjacent layer. The connections ...