book

Data Lakes

by Anne Laurent, Dominique Laurent, Cédrine Madera

June 2020

Beginner to intermediate

244 pages

5h 40m

English

Wiley-ISTE

Read now

Unlock full access

1.1. Introduction to data lakes1.2. Literature review and discussion1.3. The data lake challenges1.4. Data lakes versus decision-making systems1.5. Urbanization for data lakes1.6. Data lake functionalities1.7. Summary and concluding remarks
2.1. Introduction2.2. State of the art and practice2.3. System architecture2.4. Use case: the Constance system2.5. Concluding remarks
3.1. Our expectations3.2. Modeling data lake functionalities3.3. Building the knowledge base of industrial data lakes3.4. Our formalization approach3.5. Applying our approach3.6. Analysis of our first results3.7. Concluding remarks
4.1. Definitions and concepts4.2. Classification of metadata by NISO4.3. Other categories of metadata4.4. Sources of metadata4.5. Metadata classification4.6. Why metadata are needed4.7. Business value of metadata4.8. Metadata architecture4.9. Metadata management4.10. Metadata and data lakes4.11. Metadata management in data lakes4.12. Metadata and master data management4.13. Conclusion
5.1. Context5.2. Related work5.3. Metadata model5.4. Metadata implementation5.5. Concluding remarks
6.1. Introduction to master data management6.2. Deciding what to manage6.3. Why should I manage master data?6.4. What is master data management?6.5. Master data and the data lake6.6. Conclusion
7.1. Basic principles7.2. Using Linked Data in data lakes7.3. Limitations and issues7.4. The smart cities use case7.5. Take-home message
8.1. Introduction8.2. A little bit of context8.3. Every machine talks8.4. The volume paradox8.5. The fog, a shift in paradigm8.6. Constraint environment challenges8.7. Calculations and local drift8.8. Quality is everything8.9. Fog computing versus cloud computing and edge computing8.10. Concluding remarks: fog computing and data lake

9.1. Applying the notion of gravitation to information systems9.2. Impact of gravitation on the architecture of data lakes

Overview

The concept of a data lake is less than 10 years old, but they are already hugely implemented within large companies. Their goal is to efficiently deal with ever-growing volumes of heterogeneous data, while also facing various sophisticated user needs. However, defining and building a data lake is still a challenge, as no consensus has been reached so far. Data Lakes presents recent outcomes and trends in the field of data repositories. The main topics discussed are the data-driven architecture of a data lake; the management of metadata – supplying key information about the stored data, master data and reference data; the roles of linked data and fog computing in a data lake ecosystem; and how gravity principles apply in the context of data lakes. A variety of case studies are also presented, thus providing the reader with practical examples of data lake management.