In the Neurons section, we demonstrated that a neuron (also valid for a layer) can only classify linearly separable classes. To overcome this limitation, we have to combine multiple layers in a NN. We'll define the NN as a directed graph of operations (or layers). The graph nodes are the operations, and the edges between them determine the data flow. If two operations are connected, then the output tensor of the first will serve as input to the second, which is determined by the edge direction. A NN can have multiple inputs and outputs—the input nodes only have outgoing edges, while the outputs only have incoming edges.
Based on this definition, we can identify two main types of NNs:
- Feed-forward, which are represented by acyclic graphs. ...
