Sustainable AI

Chapter 3. Quantifying the Efficiency of Deep Learning

Recent advancements in ML/AI methods have enabled remarkable progress in multiple application domains such as computer vision, natural language processing, drug discovery, and entertainment. In particular, these advancements are due to the accelerated progress in DL that, in turn, has coincided with access to big data and large-scale compute. In this chapter, we will formalize redundancies in DL pipelines at the algorithmic and behavioral levels using the concept of AI waste, explore the compute-energy-carbon efficiency of DL, and present tools to quantify the resource efficiency of DL pipelines.

AI Waste

ML in its simplest formulation is the process of learning from data. Modern DL methods take this to another level, in terms of the volume of data and the size of models used to learn from data.¹ The data-driven approach necessitates training of overparameterized models on large datasets using some variation of the stochastic gradient descent algorithm (see “How to Train Your Model” for more details). The combination of overparameterized models, large datasets, and iterative optimization results in large-scale computations during the development and deployment of DL models. While most of these computations are necessary, there are redundant computations we can identify in DL models that do not significantly influence the downstream performance. We will refer to such redundant computations in the development and deployment ...