Redefining Data Engineering with AI

Chapter 1. Data Engineering for the Enterprise Amid AI Disruption

A Note for Early Release Readers

With Early Release ebooks, you get books in their earliest form—the author’s raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles.

This will be the 1st chapter of the final book. There is an active GitHub repository for this book.

If you’d like to be actively involved in reviewing and commenting on this draft, please reach out to the editor at [e-mail].

Enterprise organizations of the 21st century (we are a quarter of the way into it now) look quite different today than they did at the turn of the century. In fact, if you compare the list of the market leaders today with the lists from any time of the past, there are some clear and startling patterns:

• According to a study by Harvard Business Review, over 50% of the Fortune 500 list has churned out each decade from 1955 through the early 1990s, and nearly 80% turned over per decade by the late 1990s and 2000s. (Source)

• Fortune’s analysis reveals that only 49 companies have appeared on every Fortune 500 list for over 70 years. (Source)

• Only 10% of the 1955 original cohort remains on the list by 2019. One of them is IBM. (Source)

There are many reasons for this turnover, spanning from market realities to fundamentals to resilience to change. But there is one more factor that draws a clear line between pre-1995 and post-1995: the rise/evolution ...