Deep Learning with PyTorch

Optimize the neural network

We started with random weights to predict our targets and calculate loss for our algorithm. We calculate the gradients by calling the backward function on the final loss variable. This entire process repeats for one epoch, that is, for the entire set of examples. In most of the real-world examples, we will do the optimization step per iteration, which is a small subset of the total set. Once the loss is calculated, we optimize the values with the calculated gradients so that the loss reduces, which is implemented in the following function:

def optimize(learning_rate):    w.data -= learning_rate * w.grad.data    b.data -= learning_rate * b.grad.data

The learning rate is a hyper-parameter, which allows us to adjust the ...