Protocol Buffers Handbook

Book description

Delve into Google Protobuf with the help of practical examples and exercises for streamlined communication and seamless integration with gRPC

Key Features

  • Encode and decode complex data structures, enhancing data interchange efficiency across systems
  • Understand Protobuf by mastering the syntax, schema evolution, customizations, and more
  • Integrate Protobuf into your preferred language ecosystem, ensuring interoperability and effective collaboration
  • Purchase of the print or Kindle book includes a free PDF eBook

Book Description

Explore how Protocol Buffers (Protobuf) serialize structured data and provides a language-neutral, platform-neutral, and extensible solution. With this guide to mastering Protobuf, you'll build your skills to effectively serialize, transmit, and manage data across diverse platforms and languages.

This book will help you enter the world of Protocol Buffers by unraveling the intricate nuances of Protobuf syntax and showing you how to define complex data structures. As you progress, you’ll learn schema evolution, ensuring seamless compatibility as your projects evolve. The book also covers advanced topics such as custom options and plugins, allowing you to tailor validation processes to your specific requirements. You’ll understand how to automate project builds using cutting-edge tools such as Buf and Bazel, streamlining your development workflow. With hands-on projects in Go and Python programming, you’ll learn how to practically apply Protobuf concepts. Later chapters will show you how to integrate data interchange capabilities across different programming languages, enabling efficient collaboration and system interoperability.

By the end of this book, you’ll have a solid understanding of Protobuf internals, enabling you to discern when and how to use and redefine your approach to data serialization.

What you will learn

  • Focus on efficient data interchange with advanced serialization techniques
  • Master Protocol Buffers syntax and schema evolution
  • Perform custom validation via Protoc plugins for precise data integrity
  • Integrate languages seamlessly for versatile system development
  • Automate project building with Buf and Bazel
  • Get to grips with Go and Python integration for real-world Protobuf applications
  • Streamline collaboration through system interoperability with Protobuf

Who this book is for

This book is for software developers, from novices to experienced programmers, who are interested in harnessing the power of Protocol Buffers. It's particularly valuable for those seeking efficient data serialization solutions for APIs, microservices, and data-intensive applications. The content covered in this book accommodates diverse programming backgrounds, offering essential knowledge to both beginners and seasoned developers.

Table of contents

  1. Protocol Buffers Handbook
  2. Contributors
  3. About the author
  4. About the reviewer
  5. Preface
    1. Who this book is for
    2. What this book covers
    3. To get the most out of this book
    4. Download the example code files
    5. Conventions used
    6. Get in touch
    7. Share Your Thoughts
    8. Download a free PDF copy of this book
  6. Chapter 1: Serialization Primer
    1. Technical requirements
    2. Serialization’s goals
      1. How does it all work?
      2. The different data formats
    3. What about Protobuf?
      1. Serialized data size
      2. Availability of data
      3. Readability of data
      4. Type safety
      5. Readability of schema
    4. Summary
    5. Quiz
    6. Answers
  7. Chapter 2: Protobuf is a Language
    1. Technical requirements
    2. Top-level statements
      1. Syntax
      2. Edition
      3. Package
      4. Import
      5. Option
    3. User-defined types
      1. enum
    4. Message
      1. Option
      2. Field
      3. Reserved
      4. MapField
      5. Oneof
      6. Message
    5. Services
    6. Out-of-the-box types
      1. Duration and timestamp
      2. FieldMask
      3. Any
      4. Struct
    7. Summary
    8. Quiz
    9. Answers
  8. Chapter 3: Describing Data with Protobuf Text Format
    1. Technical requirements
    2. Why use the text format?
      1. Less boilerplate
      2. Type safety
      3. Headers and comments
    3. Describing data
      1. Scalar values
      2. Messages
      3. Maps
      4. Oneofs
      5. Enums
    4. Summary
    5. Quiz
    6. Answers
  9. Chapter 4: The Protobuf Compiler
    1. Technical requirements
    2. Downloading and installing protoc
      1. GitHub Releases page
      2. curl and wget
    3. Transpiling .proto files
      1. Why code generation?
      2. Generating C++ code
      3. Generating Go code
      4. How does the code map to the .proto file?
    4. Specifying import paths
    5. Encoding data to type with --encode
    6. Decoding data to type with --decode
    7. Decoding data to type without .proto files, with --decode_raw
    8. What about the other flags?
    9. Summary
    10. Quiz
    11. Answers
    12. Challenge solutions
      1. Challenge 1 – Descriptors
      2. Challenge 2 – Plugins
  10. Chapter 5: Serialization Internals
    1. Technical requirements
    2. Variable-length integers (varints)
    3. ZigZag encoding
    4. Fixed-size integers
    5. How to choose between integer types
      1. Number range
      2. Sign
      3. Data distribution
    6. Field metadata
    7. Length-delimited encoding
    8. Packed versus unpacked repeated fields
    9. Maps
    10. Summary
    11. Quiz
    12. Answers
  11. Chapter 6: Schema Evolution over Time
    1. Technical requirements
    2. Backward and forward compatibility
      1. Backward compatibility
      2. Forward compatibility
      3. Some problems
      4. Conclusion
    3. Disabling tags – reserved tags
    4. Disabling field names – reserved names
    5. The evolution rules
    6. Summary
    7. Quiz
    8. Answers
  12. Chapter 7: Implementing the Address Book in Go
    1. Technical requirements
    2. The project: address book
    3. Defining the schema
    4. Boilerplate code
      1. Converting string to enum values
      2. Reading/writing from/to a file
    5. Adding entries
      1. The business logic
      2. The CLI
    6. Listing entries
      1. The business logic
      2. The CLI
    7. Hiding sensitive data
    8. Summary
    9. Challenge
  13. Chapter 8: Implementing the Address Book in Python
    1. Technical requirements
    2. The Address Book project
    3. Defining the schema
    4. Boilerplate code
      1. Converting strings to enum values
      2. Reading/writing from/to files
    5. Adding entries
      1. The business logic
      2. The CLI
    6. Listing entries
      1. The business logic
      2. The CLI
    7. Hiding sensitive data
    8. Interoperability between Go and Python
    9. Summary
    10. Challenge
  14. Chapter 9: Developing a Protoc Plugin in Golang
    1. Technical requirements
    2. The project
    3. What are custom options?
      1. A custom option
    4. What are protoc plugins?
    5. The plugin
      1. Getting all the messages
      2. Defining a Protoc plugin
    6. Updating the AddressBook
    7. Adding a protoc plugin option
    8. Summary
    9. Challenges
  15. Chapter 10: Advanced Build
    1. Technical requirements
    2. The sample project
    3. Building manually with protoc – a summary
    4. Using Makefile
    5. Using Buf
    6. Using Bazel
    7. Summary
    8. Challenge
  16. Index
    1. Why subscribe?
  17. Other Books You May Enjoy
    1. Packt is searching for authors like you
    2. Share Your Thoughts
    3. Download a free PDF copy of this book

Product information

  • Title: Protocol Buffers Handbook
  • Author(s): Clément Jean
  • Release date: April 2024
  • Publisher(s): Packt Publishing
  • ISBN: 9781805124672