Skip to Main Content
Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow, 2nd Edition
book

Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow, 2nd Edition

by Aurélien Géron
September 2019
Intermediate to advanced content levelIntermediate to advanced
848 pages
24h 18m
English
O'Reilly Media, Inc.
Content preview from Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow, 2nd Edition

Appendix E. Other Popular ANN Architectures

In this appendix I will give a quick overview of a few historically important neural network architectures that are much less used today than deep Multilayer Perceptrons (Chapter 10), convolutional neural networks (Chapter 14), recurrent neural networks (Chapter 15), or autoencoders (Chapter 17). They are often mentioned in the literature, and some are still used in a range of applications, so it is worth knowing about them. Additionally, we will discuss deep belief nets, which were the state of the art in Deep Learning until the early 2010s. They are still the subject of very active research, so they may well come back with a vengeance in the future.

Hopfield Networks

Hopfield networks were first introduced by W. A. Little in 1974, then popularized by J. Hopfield in 1982. They are associative memory networks: you first teach them some patterns, and then when they see a new pattern they (hopefully) output the closest learned pattern. This made them useful for character recognition, in particular, before they were outperformed by other approaches: you first train the network by showing it examples of character images (each binary pixel maps to one neuron), and then when you show it a new character image, after a few iterations it outputs the closest learned character.

Hopfield networks are fully connected graphs (see Figure E-1); that is, every neuron is connected to every other neuron. Note that in the diagram the images are 6 × 6 pixels, ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Start your free trial

You might also like

Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow, 3rd Edition

Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow, 3rd Edition

Aurélien Géron
Machine Learning with PyTorch and Scikit-Learn

Machine Learning with PyTorch and Scikit-Learn

Sebastian Raschka, Yuxi (Hayden) Liu, Vahid Mirjalili

Publisher Resources

ISBN: 9781492032632Errata Page