The Cloud Data Lake

Preface

Its six in the morning; your phone gently wakes you up and automatically turns on your notifications. Your smart refrigerator reminds you that you need to order milk and shows you an option to place an order to buy more since it knows you are running low. You do that and hop on your exercise machine, where you see personalized picks based on your workout routines. You get ready and eat breakfast without bothering to look at the clock because you know your phone will tell you when it is time to start driving based on what it has learned about your commute and the traffic patterns. As you leave, your smart home assistant ensures the lights are turned off and the doors are locked. What would have seemed like science fiction a few decades ago is a regular day in our lives now. All this is possible because of the leaps technology has made in three key areas: devices that have made computing ubiquitous, connectivity that has shrunk the world by bringing the knowledge of the internet to these devices, and technology (data, artificial intelligence, machine learning) that has helped devices learn patterns and make decisions. Data is now at the heart of how the world operates, and organizations increasingly rely on data to both inform and transform their businesses.

My mind goes back to 2013, when my own personal journey with data started as I worked on identity and personalization services for Microsoft Office. It was a year of a great many learnings for me. I understood what it ...