Entity Information Life Cycle for Big Data

Entity Information Life Cycle for Big Data

by John R. Talburt, Yinle Zhou
Released April 2015
Publisher(s): Morgan Kaufmann
ISBN: 9780128006658

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

Entity Information Life Cycle for Big Data walks you through the ins and outs of managing entity information so you can successfully achieve master data management (MDM) in the era of big data. This book explains big data’s impact on MDM and the critical role of entity information management system (EIMS) in successful MDM. Expert authors Dr. John R. Talburt and Dr. Yinle Zhou provide a thorough background in the principles of managing the entity information life cycle and provide practical tips and techniques for implementing an EIMS, strategies for exploiting distributed processing to handle big data for EIMS, and examples from real applications. Additional material on the theory of EIIM and methods for assessing and evaluating EIMS performance also make this book appropriate for use as a textbook in courses on entity and identity management, data management, customer relationship management (CRM), and related topics.

  • Explains the business value and impact of entity information management system (EIMS) and directly addresses the problem of EIMS design and operation, a critical issue organizations face when implementing MDM systems
  • Offers practical guidance to help you design and build an EIM system that will successfully handle big data
  • Details how to measure and evaluate entity integrity in MDM systems and explains the principles and processes that comprise EIM
  • Provides an understanding of features and functions an EIM system should have that will assist in evaluating commercial EIM systems
  • Includes chapter review questions, exercises, tips, and free downloads of demonstrations that use the OYSTER open source EIM system
  • Executable code (Java .jar files), control scripts, and synthetic input data illustrate various aspects of CSRUD life cycle such as identity capture, identity update, and assertions

Table of contents

  1. Cover image
  2. Title page
  3. Table of Contents
  4. Copyright
  5. Foreword
  6. Preface
  7. Acknowledgements
  8. Chapter 1. The Value Proposition for MDM and Big Data
    1. Definition and Components of MDM
    2. The Business Case for MDM
    3. Dimensions of MDM
    4. The Challenge of Big Data
    5. MDM and Big Data – The N-Squared Problem
    6. Concluding Remarks
  9. Chapter 2. Entity Identity Information and the CSRUD Life Cycle Model
    1. Entities and Entity References
    2. Managing Entity Identity Information
    3. Entity Identity Information Life Cycle Management Models
    4. Concluding Remarks
  10. Chapter 3. A Deep Dive into the Capture Phase
    1. An Overview of the Capture Phase
    2. Building the Foundation
    3. Understanding the Data
    4. Data Preparation
    5. Selecting Identity Attributes
    6. Assessing ER Results
    7. Data Matching Strategies
    8. Concluding Remarks
  11. Chapter 4. Store and Share – Entity Identity Structures
    1. Entity Identity Information Management Strategies
    2. Dedicated MDM Systems
    3. The Identity Knowledge Base
    4. MDM Architectures
    5. Concluding Remarks
  12. Chapter 5. Update and Dispose Phases – Ongoing Data Stewardship
    1. Data Stewardship
    2. The Automated Update Process
    3. The Manual Update Process
    4. Asserted Resolution
    5. EIS Visualization Tools
    6. Managing Entity Identifiers
    7. Concluding Remarks
  13. Chapter 6. Resolve and Retrieve Phase – Identity Resolution
    1. Identity Resolution
    2. Identity Resolution Access Modes
    3. Confidence Scores
    4. Concluding Remarks
  14. Chapter 7. Theoretical Foundations
    1. The Fellegi-Sunter Theory of Record Linkage
    2. The Stanford Entity Resolution Framework
    3. Entity Identity Information Management
    4. Concluding Remarks
  15. Chapter 8. The Nuts and Bolts of Entity Resolution
    1. The ER Checklist
    2. Cluster-to-Cluster Classification
    3. Selecting an Appropriate Algorithm
    4. Concluding Remarks
  16. Chapter 9. Blocking
    1. Blocking
    2. Blocking by Match Key
    3. Dynamic Blocking versus Preresolution Blocking
    4. Blocking Precision and Recall
    5. Match Key Blocking for Boolean Rules
    6. Match Key Blocking for Scoring Rules
    7. Concluding Remarks
  17. Chapter 10. CSRUD for Big Data
    1. Large-Scale ER for MDM
    2. The Transitive Closure Problem
    3. Distributed, Multiple-Index, Record-Based Resolution
    4. An Iterative, Nonrecursive Algorithm for Transitive Closure
    5. Iteration Phase: Successive Closure by Reference Identifier
    6. Deduplication Phase: Final Output of Components
    7. ER Using the Null Rule
    8. The Capture Phase and IKB
    9. The Identity Update Problem
    10. Persistent Entity Identifiers
    11. The Large Component and Big Entity Problems
    12. Identity Capture and Update for Attribute-Based Resolution
    13. Concluding Remarks
  18. Chapter 11. ISO Data Quality Standards for Master Data
    1. Background
    2. Goals and Scope of the ISO 8000-110 Standard
    3. Four Major Components of the ISO 8000-110 Standard
    4. Simple and Strong Compliance with ISO 8000-110
    5. ISO 22745 Industrial Systems and Integration
    6. Beyond ISO 8000-110
    7. Concluding Remarks
  19. Appendix A. Some Commonly Used ER Comparators
  20. References
  21. Index

Product information

  • Title: Entity Information Life Cycle for Big Data
  • Author(s): John R. Talburt, Yinle Zhou
  • Release date: April 2015
  • Publisher(s): Morgan Kaufmann
  • ISBN: 9780128006658