Practical AI on the Google Cloud Platform

Chapter 4. Understanding Language on the Cloud

We have fantasized about using natural language to control computers since the original Star Trek TV series aired in the early ’60s. Characters would often interface with the computer by saying “Computer.” Since then, we have struggled to use language as an interface to computers and instead have relied entirely on symbolic languages like Python. That is, until fairly recently, with the inception of natural language systems giving us new interfaces like Siri and Alexa.

In this chapter we will discuss language and how advances in deep learning have accelerated our understanding of language. We will first look at natural language processing, or NLP, and talk about why we need it. Then we will move on to the art and science of processing language, from decimating language into numbers and vectors to processing those vectors with deep learning. After that we will discuss word context with recurrent neural networks, or RNN, and then move on to generating text with RNN layers. We will then use RNN to build sequences to sequence learning, which is at the heart of understanding machine translation. We will end with an advanced example that attempts to understand language itself using neural transformations.

Here is a high-level overview of the main topics we will cover in this chapter:

• Natural Language Processing, with Embeddings

• Recurrent Networks for NLP

• Neural Translation and the Translation API

• Natural Language API

• BERT: Bidirectional ...