Automating the Modern Data Warehouse

Chapter 3. Machine Learning, AI, and Intelligent Data Management

At one time or another, most of us who have used cloud services have marveled at the way they elide the underlying complexity of different problems and tasks. For the most part, this is a function not just of automation but, more specifically, of rule-driven automation. This kind of “smart,” rule-driven AI is not distinctive to cloud as such, and it is not necessarily new; what is different is the unprecedented scale of the cloud—not with respect to the number and capacity of its virtual hardware resources, but rather with respect to the number and variety of machines that live in it and people who use it.

The coinciding of people and machines in a single context lends itself to the purpose of studying them: that is, collecting information about and analyzing their behaviors at an unprecedented scale. This advantage is distinctive to cloud. It gives cloud providers a potentially massive dataset for training ML models and designing “AI” functions: that is, software “robots” that automate actions when they detect problems or events. These software “robots” are used to automate a range of tasks, both common and esoteric. Over time, then, cloud providers have automated a growing share of tasks and remediations.

This is radically different. There is nothing like it in the on-premises data center. Yes, much of this cloud-centered automation ultimately makes its way to the on-premises enterprise. But the automation gets ...