Machine Learning and AI with Simple Python and Matlab Scripts

Appendix AANNs with Bias

A.1 Introduction

In ANN models representing real‐life problems, a bias input is needed to adjust the weights with more accuracy and faster convergence to reduce output errors. For inclusion of a bias as part of an ANN model, one has to add an additional input at each hidden layer and the output layer of the ANN. Similar to regular ANN synapses connecting neurons, the weights of bias synapses are also associated with weights that are initially assigned random values. Bias inputs, therefore, become intermixed with link weights through summation and activation functions in forward and back propagation calculations. During training, bias weights are updated based on error values in a manner similar to synapse weights. In this appendix, we will derive the equations used in updating weights of neuron synapses and bias links in an ANN employing input biases at every iteration of training.

A.2 Training with Bias Input

A hidden‐layer neuron with a bias input is given in Figure A.1, where a bias link with weight of is added as one of the neuron inputs. Let us now formulate the weights for the incoming and outgoing links of a neuron that includes bias inputs. As shown in Figure A.2, the summation function works as follows:

Figure A.1 A neuron with bias input ...