Sustainable AI

Chapter 2. Data Parsimony

A Note for Early Release Readers

With Early Release ebooks, you get books in their earliest form—the author’s raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles.

This will be the 4th chapter of the final book. Please note that the GitHub repo will be made active later on.

If you have comments about how we might improve the content and/or examples in this book, or if you notice missing material within this chapter, please reach out to the editor at [e-mail].

Data is the new oil was a common idiom in the early 2010’s, used in the context of generating value via digital data. This phrase also unintentionally captures the increasing carbon footprint of storing and processing vast amounts of data. It is estimated that the lifecycle emissions for each TB of data on hard drive storage are anywhere between 2-20 kgCO2e per year, as illustrated for a commonly used storage devices.¹

Figure 2-1. Typical green house gas emissions across the life cycle of storage devices. Data Source: Seagate Sustainability Report

Large-scale computations on massive amounts of data have been essential to the progress in AI model development, with the most recent LLMs being trained on datasets that consist of more than 15 trillion data points (tokens).² Not all of the data used for training ...