Artificial Intelligence Programming with Python

CHAPTER 4Deep Learning

“Our intelligence is what makes us human, and AI is an extension of that quality.”

—Yann LeCun (French computer scientist)

CHAPTER OUTLINE

1. 4.1. Introduction
2. 4.2. Artificial Neural Networks
3. 4.3. Convolutional Neural Networks
4. 4.4. Recurrent Neural Networks
5. 4.5. Transformers
6. 4.6. Graph Neural Networks
7. 4.7. Bayesian Neural Networks
8. 4.8. Meta Learning
9. 4.9. Summary
10. 4.10. Chapter Review Questions

4.1 Introduction

Deep learning (DL) has attracted the world's attention since 2012, when AlexNet won the famous ImageNet challenge. Since then, deep learning has been the hottest research topic in AI. Most of the AI research news you hear today is based on deep learning.

Deep learning is largely considered a subset of machine learning, and it is built on traditional artificial neural networks. As illustrated in Figure 4.1 (left), traditional artificial neural networks typically have one input layer, one output layer, and one hidden layer. The reason they have only one hidden layer is that as the number of hidden layers increases, the complexity also increases, which makes computations unstable and impossible. Deep learning neural networks have an input layer, an output layer, and more than one layer of hidden layers. Deep learning neural networks can have more than one hidden layer due to improved algorithms and higher computing power.

Figure 4.1: Traditional ...