Deep Learning Essentials

Origins of CNNs

Walter Pitts and Warren McCulloch are often credited with the first computer model in 1943, which was inspired by the neural network-based structure of the human brain. They proposed a technique that inspired the notion of logic-based design and provided a formalism under which future refinements led to the invention of Finite Automata. The McCulloch-Pitts network was a directed graph where each node was a neuron and edges were marked as either excitatory (1) or inhibitory (0), and used a threshold logic to replicate the human thought process.

One of the challenges in this design was the learning of thresholds or weights, as would be defined later. Henry J. Kelley provided the first version of this learning algorithm in the ...