Federated Learning

Chapter 12: Hyperparameter tuning for federated learning – systems and practices

Syed Zawad^a; Feng Yan^b    aUniversity of Nevada, Reno, NV, United States^bUniversity of Houston, Houston, TX, United States

Abstract

Hyperparameter tuning is a time and resource demanding task. Compared to conventional centralized training, federated learning faces several unique challenges. First, in federated learning (especially in the cross-device paradigm), training is often conducted in unreliable (e.g., clients often drop during training) and resource constrained devices. In addition, federated learning often introduces additional hyperparameters, such as the number of local epochs, which requires even more tuning time and resources. Furthermore, both data and ...