Understanding Log Analytics at Scale, 2nd Edition

Chapter 7. Enabling Log Data’s Strategic Value with a Unified Fast File and Object Platform

Even though data is widely regarded today as every business’s most important asset, it tends to be isolated in silos across the enterprise that are each developed to meet the needs of a discrete set of applications and workloads. This fragmentation of data limits the degree to which it can be accessed in real time, hampering the ability to perform flexible analytics on it. This limitation runs counter to the modern perspective of data as a primary strategic differentiator, unleashed by the power of analytics.

Maintaining architectures that are customized for specific workloads or applications expand the number of data silos. Data warehouses have been built as a single repository for integrating and storing structured data pulled from multiple unstructured data sources across your organization. However, they have historically been used for batch processing of structured data and often use an architecture that is customized for a specific data warehouse application.

Alternatively, data lakes are essentially uncurated masses of data, dwelling in a storage architecture designed to store its contents as efficiently as possible, rather than with speed of access, sharing, and delivery. This property makes the data-lake approach to storage limited for log analytics. In addition, data lakes lack the ability to tailor data delivery to the specific latency, throughput, and I/O requirements of individual ...