Hands-On Generative Adversarial Networks with PyTorch 1.x

Parameter initialization

Sometimes, one of the most frustrating things about learning about an optimization method from a book/paper and implementing it with code is that the initial state of the machine learning system (initial values of the parameters) can have a great impact on the model's final performance. It is important to have knowledge of parameter initialization, especially while you're dealing with deep networks. A good parameter initialization also means that you won't always rely on Batch Normalization to keep your parameters in line during training. To quote from the PyTorch documentation, "A PyTorch Tensor is basically the same as a numpy array: it does not know anything about deep learning or computational graphs or gradients ...