Building Big Data Applications

Building Big Data Applications

by Krish Krishnan
Released November 2019
Publisher(s): Academic Press
ISBN: 9780128158043

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

Building Big Data Applications helps data managers and their organizations make the most of unstructured data with an existing data warehouse. It provides readers with what they need to know to make sense of how Big Data fits into the world of Data Warehousing. Readers will learn about infrastructure options and integration and come away with a solid understanding on how to leverage various architectures for integration. The book includes a wide range of use cases that will help data managers visualize reference architectures in the context of specific industries (healthcare, big oil, transportation, software, etc.).

  • Explores various ways to leverage Big Data by effectively integrating it into the data warehouse
  • Includes real-world case studies which clearly demonstrate Big Data technologies
  • Provides insights on how to optimize current data warehouse infrastructure and integrate newer infrastructure matching data processing workloads and requirements

Table of contents

  1. Cover image
  2. Title page
  3. Table of Contents
  4. Copyright
  5. Dedication
  6. Preface
  7. 1. Big Data introduction
    1. Big Data delivers business value
    2. Big Data applications—processing data
    3. Critical factors for success
    4. Risks and pitfalls
  8. 2. Infrastructure and technology
    1. Introduction
    2. Distributed data processing
    3. Big data processing requirements
    4. Technologies for big data processing
    5. MapReduce
    6. MapReduce programming model
    7. MapReduce Google architecture
    8. History
    9. Hadoop core components
    10. NameNode
    11. DataNode
    12. Image
    13. Journal
    14. Checkpoint
    15. HDFS startup
    16. Block allocation and storage
    17. HDFS client
    18. Replication and recovery
    19. NameNode and DataNode—communication and management
    20. Heartbeats
    21. CheckPointNode and BackupNode
    22. CheckPointNode
    23. BackupNode
    24. Filesystem snapshots
    25. YARN scalability
    26. YARN execution flow
    27. Zookeeper features
    28. Locks and processing
    29. Failure and recovery
    30. Programming with Pig Latin
    31. Pig data types
    32. Running Pig programs
    33. Pig program flow
    34. Common Pig command
    35. HBASE architecture
    36. HBASE architecture implementation
    37. Hive architecture
    38. Execution—how does Hive process queries?
    39. Hive data types
    40. Hive examples
    41. HCatalog
    42. CAP theorem
    43. A keyspace has configurable properties that are critical to understand
    44. Cassandra ring architecture
    45. The design features of document-oriented databases include the following:
  9. 3. Building big data applications
    1. Data storyboard
  10. 4. Scientific research applications and usage
    1. Accelerators
    2. Big data platform and application
    3. XRootD filesystem interface project
    4. Service for web-based analysis (SWAN)
    5. The result—Higgs Boson discovery
  11. 5. Pharmacy industry applications and usage
    1. The complexity design for data applications
    2. Complexities in transformation of data
    3. Google deep mind
    4. Case study
  12. 6. Visualization, storyboarding and applications
    1. Let us look at some of the use cases of big data applications
    2. Visualization
    3. The evolving role of the data scientist
  13. 7. Banking industry applications and usage
    1. The coming of age with uber banking
    2. The use cases of analytics and big data applications in banking today
    3. Fraud and compliance tracking
    4. Client chatbots for call center
    5. Antimoney laundering detection
    6. Algorithmic trading
    7. Recommendation engines
  14. 8. Travel and tourism industry applications and usage
    1. Travel and big data
    2. Real-time conversion optimization
    3. Optimized disruption management
    4. Niche targeting and unique selling propositions
    5. “Smart” social media listening and sentiment analysis
    6. Hospitality industry and big data
    7. Analytics and travel industry
    8. Examples of the use of predictive analytics
    9. Develop applications using data and agile API
  15. 9. Governance
    1. Definition
    2. Metadata and master data
    3. Master data
    4. Data management in big data infrastructure
    5. Processing complexity of big data
    6. Processing limitations
    7. Governance model for building an application
    8. Use cases of governance
  16. 10. Building the big data application
    1. Risk assessment questions
    2. Business continuity management
  17. 11. Data discovery and connectivity
    1. Challenges before you start with AI
    2. Strategies you can follow to start with AI
    3. Compliance and regulations
  18. Use cases from industry vendors
  19. Index

Product information

  • Title: Building Big Data Applications
  • Author(s): Krish Krishnan
  • Release date: November 2019
  • Publisher(s): Academic Press
  • ISBN: 9780128158043