Programming Neural Networks with Python

4    Learning in a Simple Network

What does learning in a neural network mean? We’ll describe learning in this context and what that looks like in concrete terms. You’ll also see that you can learn in more than one way!

This chapter takes us back to the beginnings of learning for artificial neural networks (ANNs). In 1949, Donald Hebb discovered the effect that the connection between two neurons is strengthened if they fire simultaneously, which is the basic mechanism for learning and memory. Motivated by these discoveries, a directional learning strategy emerged called perceptron learning, which explained learning for datasets that are cleanly separable. Encouraged by the success of this learning approach and aware of its shortcomings, the next ...