Natural Language Processing with Spark NLP

Chapter 4. Deep Learning Basics

In this chapter we will cover the basics of deep learning. The goal of this chapter is to create a foundation for us to discuss how to apply deep learning to NLP. There are new deep learning techniques being developed every month, and we will cover some of the newer techniques in later chapters, which is why we need this foundation. In the beginning of this chapter we will cover some of the history of the artificial neural network, and we will work through some example networks representing logical operators. This will help us build a solid foundation for thinking about artificial neural networks.

Fundamentally, deep learning is a field of study of artificial neural networks, or ANNs. The first appearance of artificial neural networks in academic literature was in a paper called A Logical Calculus of the Ideas Immanent in Nervous Activity, by Warren S. McCulloch and Walter Pitts in 1943. Their work was an attempt to explain how the brain worked from a cyberneticist perspective. Their work would become the root of modern neuroscience and modern artificial neural networks.

An ANN is a biologically inspired algorithm. ANNs are not realistic representations of how a brain learns, although from time to time news stories still hype this. We are still learning many things about how the brain processes information. As new discoveries are made, there is often an attempt to represent real neurological structures and processes in terms of ANNs, like the concept ...