Hands-On Generative Adversarial Networks with Keras

Sensitivity to hyperparameter initialization

In their paper Are GANs Created Equal? A Large-Scale Study, researchers at Google Brain did a large-scale study in which multiple GAN losses with different hyperparameter initializations were evaluated under multiple evaluation measures. In this study, they found that most models can reach similar scores with enough hyperparameter optimization and random restarts. This suggests that most models are very sensitive to hyperparameter initialization, and that improvements can arise from computational budgeting and tuning rather than algorithmic changes.