Data Lake Development with Big Data

Data Discovery and metadata

Data Discovery deals with the identification of related data assets, making them discoverable and guiding the data consumers to relevant datasets.

The efficiency of Data Discovery depends upon the amount and quality of the metadata that is captured as the data moves across the various tiers in the Data Lake. Metadata keeps track of all the data assets that reside on a Data Lake; it helps data consumers to find the relevant data. Metadata identifies and maintains relationships between data, right from the time the data is ingested, enhanced, transformed, and evolved. It guides consumers to related datasets that can be combined and integrated.

Semantic metadata captures the semantics of the data; semantics is the ability ...