Software Architecture for Big Data and the Cloud

Software Architecture for Big Data and the Cloud

by Ivan Mistrik, Rami Bahsoon, Nour Ali, Maritta Heisel, Bruce Maxim
Released June 2017
Publisher(s): Morgan Kaufmann
ISBN: 9780128093382

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

Software Architecture for Big Data and the Cloud is designed to be a single resource that brings together research on how software architectures can solve the challenges imposed by building big data software systems. The challenges of big data on the software architecture can relate to scale, security, integrity, performance, concurrency, parallelism, and dependability, amongst others. Big data handling requires rethinking architectural solutions to meet functional and non-functional requirements related to volume, variety and velocity.

The book's editors have varied and complementary backgrounds in requirements and architecture, specifically in software architectures for cloud and big data, as well as expertise in software engineering for cloud and big data. This book brings together work across different disciplines in software engineering, including work expanded from conference tracks and workshops led by the editors.

  • Discusses systematic and disciplined approaches to building software architectures for cloud and big data with state-of-the-art methods and techniques
  • Presents case studies involving enterprise, business, and government service deployment of big data applications
  • Shares guidance on theory, frameworks, methodologies, and architecture for cloud and big data

Table of contents

  1. Cover image
  2. Title page
  3. Table of Contents
  4. Copyright
  5. Contributors
  6. About the Editors
  7. Foreword by Mandy Chessell
    1. Amnesia or Progress?
  8. Foreword by Ian Gorton
  9. Preface
    1. Introduction
    2. Why a New Book on Software Architecture for Big Data and the Cloud?
    3. Book Outline
    4. Part I: Concepts and Models
    5. Part II: Analyzing and Evaluating
    6. Part III: Technologies
    7. Part IV: Resource Management
    8. Part V: Looking Ahead
  10. Chapter 1: Introduction. Software Architecture for Cloud and Big Data: An Open Quest for the Architecturally Significant Requirements
    1. Abstract
    2. 1.1. A Perspective into Software Architecture for Cloud and Big Data
    3. 1.2. Cloud Architecturally Significant Requirements and Their Design Implications
    4. 1.3. Big Data Management as Cloud Architecturally Significant Requirement
    5. References
  11. Part 1: Concepts and Models
    1. Chapter 2: Hyperscalability – The Changing Face of Software Architecture
      1. Abstract
      2. 2.1. Introduction
      3. 2.2. Hyperscalable Systems
      4. 2.3. Principles of Hyperscalable Systems
      5. 2.4. Related Work
      6. 2.5. Conclusions
      7. References
    2. Chapter 3: Architecting to Deliver Value From a Big Data and Hybrid Cloud Architecture
      1. Abstract
      2. 3.1. Introduction
      3. 3.2. Supporting the Analytics Lifecycle
      4. 3.3. The Role of Data Lakes
      5. 3.4. Key Design Features That Make a Data Lake Successful
      6. 3.5. Architecture Example – Context Management in the IoT
      7. 3.6. Big Data Origins and Characteristics
      8. 3.7. The Systems That Capture and Process Big Data
      9. 3.8. Operating Across Organizational Silos
      10. 3.9. Architecture Example – Local Processing of Big Data
      11. 3.10. Architecture Example – Creating a Multichannel View
      12. 3.11. Application Independent Data
      13. 3.12. Metadata and Governance
      14. 3.13. Conclusions
      15. 3.14. Outlook and Future Directions
      16. References
    3. Chapter 4: Domain-Driven Design of Big Data Systems Based on a Reference Architecture
      1. Abstract
      2. 4.1. Introduction
      3. 4.2. Domain-Driven Design Approach
      4. 4.3. Related Work
      5. 4.4. Feature Model of Big Data Systems
      6. 4.5. Deriving the Application Architectures and Example
      7. 4.6. Conclusion
      8. References
    4. Chapter 5: An Architectural Model-Based Approach to Quality-Aware DevOps in Cloud Applicationsc
      1. Abstract
      2. 5.1. Introduction
      3. 5.2. A Cloud-Based Software Application
      4. 5.3. Differences in Architectural Models Among Development and Operations
      5. 5.4. The iObserve Approach
      6. 5.5. Addressing the Differences in Architectural Models
      7. 5.6. Applying iObserve to CoCoME
      8. 5.7. Limitations
      9. 5.8. Related Work
      10. 5.9. Conclusion
      11. References
    5. Chapter 6: Bridging Ecology and Cloud: Transposing Ecological Perspective to Enable Better Cloud Autoscaling
      1. Abstract
      2. Acknowledgement
      3. 6.1. Introduction
      4. 6.2. Motivation
      5. 6.3. Natural Ecosystem
      6. 6.4. Transposing Ecological Principles, Theories and Models to Cloud Ecosystem
      7. 6.5. Ecology-Inspired Self-Aware Pattern
      8. 6.6. Opportunities and Challenges
      9. 6.7. Related Work
      10. 6.8. Conclusion
      11. References
  12. Part 2: Analyzing and Evaluating
    1. Chapter 7: Evaluating Web PKIs
      1. Abstract
      2. 7.1. Introduction
      3. 7.2. An Overview of PKI
      4. 7.3. Desired Features and Security Concerns
      5. 7.4. Existing Proposals
      6. 7.5. Observations
      7. 7.6. Conclusion
      8. References
    2. Chapter 8: Performance Isolation in Cloud-Based Big Data Architectures
      1. Abstract
      2. 8.1. Introduction
      3. 8.2. Background
      4. 8.3. Case Study and Problem Statement
      5. 8.4. Performance Monitoring in Cloud-Based Systems
      6. 8.5. Application Framework for Performance Isolation
      7. 8.6. Evaluation of the Framework
      8. 8.7. Discussion
      9. 8.8. Related Work
      10. 8.9. Conclusion
      11. References
    3. Chapter 9: From Legacy to Cloud: Risks and Benefits in Software Cloud Migration
      1. Abstract
      2. 9.1. Introduction
      3. 9.2. Research Method
      4. 9.3. Results
      5. 9.4. Discussion
      6. 9.5. Conclusion
      7. References
    4. Chapter 10: Big Data: A Practitioners Perspective
      1. Abstract
      2. 10.1. Big Data Is a New Paradigm – Differences With Traditional Data Warehouse, Pitfalls and Consideration
      3. 10.2. Product Considerations for Big Data – Use of Open Source Products for Big Data, Pitfalls and Considerations
      4. 10.3. Use of Cloud for hosting Big Data – Why to Use Cloud, Pitfalls and Consideration
      5. 10.4. Big Data Implementation – Architecture Definition, Processing Framework and Migration Pattern From Data Warehouse to Big Data
      6. 10.5. Conclusion
      7. References
  13. Part 3: Technologies
    1. Chapter 11: A Taxonomy and Survey of Stream Processing Systems
      1. Abstract
      2. 11.1. Introduction
      3. 11.2. Stream Processing Platforms: A Brief Background
      4. 11.3. Taxonomy
      5. 11.4. A Survey of Stream Processing Platforms
      6. 11.5. Comparison Study of the Stream Processing Platforms
      7. 11.6. Conclusions and Future Directions
      8. References
    2. Chapter 12: Architecting Cloud Services for the Digital Me in a Privacy-Aware Environment
      1. Abstract
      2. 12.1. Introduction
      3. 12.2. Example
      4. 12.3. Challenges
      5. 12.4. Preliminaries
      6. 12.5. System-of-Systems Approach
      7. 12.6. Generative Approach
      8. 12.7. Related Work
      9. 12.8. Discussion
      10. 12.9. Conclusion
      11. References
    3. Chapter 13: Reengineering Data-Centric Information Systems for the Cloud – A Method and Architectural Patterns Promoting Multitenancy
      1. Abstract
      2. 13.1. Introduction
      3. 13.2. Context and Problem: Multitenancy in Cloud Computing
      4. 13.3. Solution Overview: Reengineering Method and Process
      5. 13.4. Solution Detail 1: Architectural Patterns in the Method
      6. 13.5. Solution Detail 2: Testing and Code Reviews
      7. 13.6. Case Study (Implementation)
      8. 13.7. Discussion
      9. 13.8. Related Work
      10. 13.9. Summary and Conclusions
      11. Appendix 13.A. Architectural Refactoring (AR) Reference
      12. References
    4. Chapter 14: Exploring the Evolution of Big Data Technologies
      1. Abstract
      2. 14.1. Introduction
      3. 14.2. Big Data in Our Daily Lives
      4. 14.3. Data Intensive Computing
      5. 14.4. Apache Hadoop
      6. 14.5. Apache Spark
      7. 14.6. The Role of Cloud Computing
      8. 14.7. The Future of Big Data Platforms
      9. 14.8. Conclusion
      10. References
    5. Chapter 15: A Taxonomy and Survey of Fault-Tolerant Workflow Management Systems in Cloud and Distributed Computing Environments
      1. Abstract
      2. 15.1. Introduction
      3. 15.2. Background
      4. 15.3. Introduction to Fault-Tolerance
      5. 15.4. Taxonomy of Faults
      6. 15.5. Taxonomy of Fault-Tolerant Scheduling Algorithms
      7. 15.6. Modeling of Failures in Workflow Management Systems
      8. 15.7. Metrics Used to Quantify Fault-Tolerance
      9. 15.8. Survey of Workflow Management Systems and Frameworks
      10. 15.9. Tools and Support Systems
      11. 15.10. Summary
      12. References
  14. Part 4: Resource Management
    1. Chapter 16: The HARNESS Platform: A Hardware- and Network-Enhanced Software System for Cloud Computing
      1. Abstract
      2. Acknowledgements
      3. 16.1. Introduction
      4. 16.2. Related Work
      5. 16.3. Overview
      6. 16.4. Managing Heterogeneity
      7. 16.5. Prototype Description
      8. 16.6. Evaluation
      9. 16.7. Conclusion
      10. Project Resources
      11. References
    2. Chapter 17: Auditable Version Control Systems in Untrusted Public Clouds
      1. Abstract
      2. 17.1. Motivation and Contributions
      3. 17.2. Background Knowledge
      4. 17.3. System and Adversarial Model
      5. 17.4. Auditable Version Control Systems
      6. 17.5. Discussion
      7. 17.6. Other RDIC Approaches for Version Control Systems
      8. 17.7. Evaluation
      9. 17.8. Conclusion
      10. References
    3. Chapter 18: Scientific Workflow Management System for Clouds
      1. Abstract
      2. 18.1. Introduction
      3. 18.2. Background
      4. 18.3. Workflow Management Systems for Clouds
      5. 18.4. Cloudbus Workflow Management System
      6. 18.5. Cloud-Based Extensions to the Workflow Engine
      7. 18.6. Performance Evaluation
      8. 18.7. Summary and Conclusions
      9. References
  15. Part 5: Looking Ahead
    1. Chapter 19: Outlook and Future Directions
      1. Abstract
      2. 19.1. New or Advanced Applications
      3. 19.2. Advanced Supporting Technologies
      4. 19.3. Architecturally Significant Requirements
      5. 19.4. Challenges for the Architecting Process
      6. 19.5. Further Reading
      7. References
  16. Glossary
  17. Author Index
  18. Subject Index

Product information

  • Title: Software Architecture for Big Data and the Cloud
  • Author(s): Ivan Mistrik, Rami Bahsoon, Nour Ali, Maritta Heisel, Bruce Maxim
  • Release date: June 2017
  • Publisher(s): Morgan Kaufmann
  • ISBN: 9780128093382