The Enterprise Big Data Lake

Chapter 2. Historical Perspective

Data has been around for a very long time. First, it was visual: paintings on cave walls, mounds arranged a certain way. Once humans developed systems of writing, they used these to keep track of things: many ancient clay tablets and manuscripts seem to contain inventories, ledgers, and bills of sale and debt. Later, more general data was collected and published in almanacs and encyclopedias: records of the longest rivers, tallest mountains, deepest lakes, most populous countries, average rainfall, highest and lowest temperatures, and so on. We seem to have an endless fascination with measuring and counting, comparing and tracking. Traditionally, this measuring and counting process was laborious and manual, so we invented machines to help us with it. Eventually, those machines evolved into modern computers.

Very early on, it became obvious that computers had a capacity to count, measure, and store information that far exceeded a human’s. However, computers were also great at other things, like applying logic and executing business processes. Most of the focus in the early days of computers was on programs and logic. Data was considered an artifact of programs, something that could be accessed and made sense of only by the programs that had originally stored it.

To make data humanly accessible, programmers developed reports that packaged data into a human-readable form. If an analyst wanted to look at data in a different way, they had to make a ...