What's New In TensorFlow 2.x?

Appendix A. Introduction to Deep Learning

This appendix introduces deep learning, in case you are not familiar with this topic. It is recommended that you read this before proceeding with the report because we require an understanding of the concepts described in this chapter. We assume that you are familiar with linear regression and logistic regression though.

Understanding Deep Learning

Before we learn about deep learning, we need to understand artificial neural networks. Let’s start with the most simple neural network, the so-called perceptron.

Tip

An artificial neuron can be seen as applying the result of linear regression to an activation function.

The Perceptron

The perceptron, invented in 1958 and first implemented in software on the IBM 704 computer, is the most basic functional unit that is loosely inspired by a biological neuron. A neuron is a basic mathematical functional unit that computes a weighted sum on its inputs and applies an activation function on the result. As illustrated in Figure A-1, inputs $x_1,x_2,x_3$ are multiplied with weights $w_1,w_2,w_3$.

Figure A-1. The perceptron

Then, the results of those multiplications are summed and added to $w_0$. Finally, an activation function is applied, to add a nonlinear component to the preceding linear-only computation. In the perceptron, it is a simple step or threshold function, defined as ...