Machine Learning and AI with Simple Python and Matlab Scripts

2Artificial Neural Networks

2.1 Introduction

Biological nervous systems process information using nerve cells called neurons that transmit messages to other cells using connections referred to as synapses. Information in the brain is stored by strengthening synapses among a group of neurons through a reinforcement process that repeatedly interchanges messages among these cells. Neurons operate in groups and react to inputs applied to them by further propagating these inputs to other neurons only over the strengthened synapses. No single neuron stores any particular piece of information, but a group of neurons together with established synapses represent information in the brain [23].

Brain‐inspired systems, called artificial neural networks (ANNs), intend to replicate the way that humans learn. ANNs are built of nodes (i.e. artificial neurons) which are interconnected by directed links (i.e. synapses) representing a connection from the output of one artificial neuron to the input of another. Each link in this system is associated with a weight, which alters the information as it travels from one neuron to another. In an ANN, nodes are organized in layers responsible for processing information in sequence. Signals are processed starting from the input layer, passing intermediate hidden layers until reaching the output layer, which generates observable outputs. Figure 2.1 shows an example of an ANN consisting of an input, two hidden and one output layers. Note that an engineer ...