Basically, logistic regression is a feed-forward neural network without a hidden layer, where the output layer directly connects with the input layer. In other words, logistic regression is a single neuron that maps the input to the output layer. Theoretically, the neural networks with an additional hidden layer between the input and output layer should be able to learn more about the relationship underneath.
A single-layer neural network for two possible classes can be represented graphically as follows:
Suppose x is n-dimension, and there are H hidden units in the hidden layer, ...
