Sustainable AI

Chapter 2. Under the Hood of Generative AI

In the 2004 science-fiction movie I, Robot, there is a memorable scene where the human protagonist derisively asks a humanoid robot, “Can a robot write a symphony? Can a robot turn a canvas into a beautiful masterpiece?” The robot responds, “Can you?” and the protagonist’s face turns pale in response.

We have considered the arts such as creative writing, painting, and composing music as the highest feats of human intelligence. We have ascribed machines to be good at more “mechanical” tasks like crunching big numbers or doing repetitive tasks. That is, until the recent revolution of generative AI (GenAI), which has brought us to fundamentally question what it means to create. Models like ChatGPT can generate text,¹ Stable Diffusion can generate images in artistic styles,² and MusicLM can generate songs.³ Whether the output of these GenAI models is comparable to human creativity is being disputed from both technical and philosophical standpoints, as it should be.⁴ In this book we’re concerned with the material resources necessary to develop AI models like the ones behind GenAI, and their impact on sustainability.⁵

To assess the environmental sustainability of AI models, including GenAI, we first need to understand these models. And fundamental concepts from ML are essential to do this. Furthermore, improving the resource efficiency of AI models requires a careful analysis of several underlying technical phenomena. The objective of this ...