Hands-On Generative Adversarial Networks with Keras

Normalization in the generator and the discriminator

In their paper, Tero Karras et al. describe an unexpected approach for weight initialization that goes against the current trend of carefully initializing the weights of GAN models. Their approach is simple and fast: they initialize the weights from a normal distribution with a mean of 0 and a standard deviation of 1, and then scale the weights using a per-layer normalization constant. That is, given the weight and the per-layer normalization constant, the weights become .