Database Design and Modeling with Google Cloud

Database Design and Modeling with Google Cloud

by Abirami Sukumaran
Released December 2023
Publisher(s): Packt Publishing
ISBN: 9781804611456

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

Build faster and efficient real-world applications on the cloud with a fitting database model that's perfect for your needs

Key Features

  • Familiarize yourself with business and technical considerations involved in modeling the right database
  • Take your data to applications, analytics, and AI with real-world examples
  • Learn how to code, build, and deploy end-to-end solutions with expert advice
  • Purchase of the print or Kindle book includes a free PDF eBook

Book Description

In the age of lightning-speed delivery, customers want everything developed, built, and delivered at high speed and at scale. Knowledge, design, and choice of database is critical in that journey, but there is no one-size-fits-all solution. This book serves as a comprehensive and practical guide for data professionals who want to design and model their databases efficiently.

The book begins by taking you through business, technical, and design considerations for databases. Next, it takes you on an immersive structured database deep dive for both transactional and analytical real-world use cases using Cloud SQL, Spanner, and BigQuery. As you progress, you’ll explore semi-structured and unstructured database considerations with practical applications using Firestore, cloud storage, and more. You’ll also find insights into operational considerations for databases and the database design journey for taking your data to AI with Vertex AI APIs and generative AI examples.

By the end of this book, you will be well-versed in designing and modeling data and databases for your applications using Google Cloud.

What you will learn

  • Understand different use cases and real-world applications of data in the cloud
  • Work with document and indexed NoSQL databases
  • Get to grips with modeling considerations for analytics, AI, and ML
  • Use real-world examples to learn about ETL services
  • Design structured, semi-structured, and unstructured data for your applications and analytics
  • Improve observability, performance, security, scalability, latency SLAs, SLIs, and SLOs

Who this book is for

This book is for database developers, data engineers, and architects looking to design, model, and build database applications on the cloud with an extended focus on operational consideration and taking their data to AI. Data scientists, as well ML and AI engineers who want to use Google Cloud services in the data to AI journey will also find plenty of useful information in this book. It will also be useful to data analysts and BI developers who want to use SQL impactfully to generate ML and generative AI insights from their data.

Table of contents

  1. Database Design and Modeling with Google Cloud
  2. Forewords
  3. Contributors
  4. About the author
  5. About the reviewers
  6. Preface
    1. Who this book is for
    2. What this book covers
    3. To get the most out of this book
    4. Download the example code files
    5. Conventions used
    6. Get in touch
    7. Share Your Thoughts
    8. Download a free PDF copy of this book
  7. Part 1:Database Model: Business and Technical Design Considerations
  8. Chapter 1: Data, Databases, and Design
    1. Data
    2. Databases
      1. A teeny-tiny bit about the evolution of databases
      2. DBMS
    3. Database design
      1. Data modeling
      2. Database modeling
    4. Considerations for a good database design
    5. Business aspect
      1. Ingestion
    6. Technical aspect
    7. Choosing the right database
      1. Relational database
      2. NoSQL database
    8. Summary
  9. Chapter 2: Handling Data on the Cloud
    1. Types of cloud services
    2. Use case categories
    3. The benefits of cloud computing
    4. Data applications on cloud
      1. Storage
      2. Backup and disaster recovery
      3. Analytics and insights
      4. Application development
      5. User experience and personalization
    5. Managed, unmanaged, and database as a service
      1. Managed databases
      2. Unmanaged databases
      3. Database as a service
    6. Cloud database considerations
    7. A quick follow-up
    8. Summary
  10. Part 2:Structured Data
  11. Chapter 3: Database Modeling for Structured Data
    1. Structured data
      1. Rows and columns
      2. Transactional applications
      3. Analytical applications
    2. Using an RDBMS for structured data
      1. Atomicity
      2. Consistency
      3. Isolation
      4. Durability
    3. Considerations for your RDBMS
    4. Structured query language
      1. Sample SQL queries
    5. Summary
  12. Chapter 4: Setting Up a Fully Managed RDBMS
    1. Fully managed databases
    2. Fully managed RDBMS
    3. Cloud SQL
    4. Setting up and configuring a fully managed RDBMS
      1. Creating a Cloud SQL instance for MySQL
      2. Connecting to the instance
      3. Creating a database
      4. Creating a table
      5. Inserting values
      6. Querying values
    5. Creating an application with the Cloud database
      1. Configuring the Cloud Functions service account
      2. Creating a Cloud Function
    6. Operational aspects of cloud relational databases
      1. Migration
      2. Monitoring
      3. Query Insights
      4. Security
    7. Summary
  13. Chapter 5: Designing an Analytical Data Warehouse
    1. Understanding how data warehouses are different from databases
    2. Significance of ETL in data warehouse
    3. Learning about BigQuery
      1. Features of BigQuery
      2. Setting up and configuring a fully managed data warehouse with BigQuery
      3. Enabling BigQuery from the console
      4. Creating a BigQuery dataset
      5. Using an existing public dataset
      6. Creating a table in the dataset
      7. Performing simple analytics
    4. Summary of operational aspects and design considerations
    5. Summary
  14. Part 3:Semi-Structured, Unstructured Data, and NoSQL Design
  15. Chapter 6: Designing for Semi-Structured Data
    1. Semi-structured data
      1. Pros and cons of semi-structured data
      2. Use cases of semi-structured data
    2. NoSQL for semi-structured data
      1. Data structures supported by NoSQL databases
    3. Firestore and its features
    4. Setting up Firestore
      1. Collection
      2. Document
      3. Subcollection
    5. Security
    6. Client libraries and APIs
    7. Indexing
      1. Single-field index
      2. Composite index
      3. Collection group query
    8. Data model considerations
      1. Hierarchical format
      2. Denormalized format
    9. Easy querying with RunQuery API
      1. API endpoint and method
      2. The parent parameter
      3. JSON body format
      4. StructuredQuery
      5. The from clause
      6. The where clause
      7. Putting the pieces together
    10. Implementing RunQuery API programmatically
    11. Summary
  16. Chapter 7: Unstructured Data Management
    1. Use cases
      1. Processing unstructured data
    2. Storage options in Google Cloud
      1. Cloud Storage, classes, and features
    3. Unstructured data storage with BigQuery
      1. External sources
      2. External connections
    4. Unstructured data analytics with BigQuery
    5. Summary
  17. Part 4:DevOps and Databases
  18. Chapter 8: DevOps and Databases
    1. Upgrades, updates, and patching
    2. Security, privacy, and encryption
    3. Replication and availability
    4. Scalability
    5. Performance and throughput
    6. SLA, SLI, and SLO
    7. Data federation
    8. Continuous integration/continuous delivery (CI/CD)
    9. Migrating to cloud databases
    10. Database Migration Service
    11. System, query, and performance insights
    12. Summary
  19. Part 5:Data to AI
  20. Chapter 9: Data to AI – Modeling Your Databases for Analytics and ML
    1. Modeling considerations for analytics, AI, and ML
    2. Data to AI
    3. Google Cloud ETL services
    4. Google Cloud Dataflow at a glance
      1. Real-world use cases for Google Cloud Dataflow
      2. Step-by-step guide to Google Cloud Dataflow
    5. Taking your data to AI
    6. Summary
  21. Chapter 10: Looking Ahead – Designing for LLM Applications
    1. Capturing the evolution of LLMs
    2. Getting started with LLMs
      1. Understanding the underlying principles of LLMs
    3. Comparing real-world applications of LLMs and traditional analytics
    4. Understanding the differences in data modeling for traditional analytics and LLMs
    5. Data model design considerations for applications that use LLMs
    6. Learning about data modeling principles and techniques
    7. Ethical and responsible practices
    8. Hands-on time – building an LLM application
      1. Step 1 – create a table
      2. Step 2 – insert data into the table
      3. Step 3 – create an external connection for BigQuery to access the Vertex AI model
      4. Step 4 – grant permissions to the service account to access the Vertex AI service
      5. Step 5 – create the remote model in BigQuery
      6. Step 6 – query the dataset
      7. Step 7 – generate text (create an LLM application) using only SQL
    9. Vector databases
    10. Summary
    11. Onward and upward!
  22. Index
    1. Why subscribe?
  23. Other Books You May Enjoy
    1. Packt is searching for authors like you
    2. Share Your Thoughts
    3. Download a free PDF copy of this book

Product information

  • Title: Database Design and Modeling with Google Cloud
  • Author(s): Abirami Sukumaran
  • Release date: December 2023
  • Publisher(s): Packt Publishing
  • ISBN: 9781804611456