In the Neurons section, we demonstrated that a neuron (also valid for a layer) can only classify linearly separable classes. To overcome this limitation, we have to combine multiple layers in a NN. We'll define the NN as a directed graph of operations (or layers). The graph nodes are the operations, and the edges between them determine the data flow. If two operations are connected, then the output tensor of the first will serve as input to the second, which is determined by the edge direction. A NN can have multiple inputs and outputsthe input nodes only have outgoing edges, while the outputs only have incoming edges.

Based on this definition, we can identify two main types of NNs:

  • Feed-forward, which are represented by acyclic graphs.  ...

Get Advanced Deep Learning with Python now with O’Reilly online learning.

O’Reilly members experience live online training, plus books, videos, and digital content from 200+ publishers.