Principles and Practice of Big Data, 2nd Edition

6

Introspection

Abstract

Big Data resources are highly complex; their data cannot be organized as a simple array of rows and columns. Consequently, the data in Big Data resources must be annotated with information that describes the contained data. The most practical way of associating an object with the data that describes the object is through encapsulation, wherein each data object carries all the information necessary to describe itself. The interrogation of a data object for the information it contains is called introspection. Introspection is one of the most important features in a well-designed Big Data resource. Without introspection it is nearly impossible to determine whether the data extracted from a query is relevant (i.e., has ...