O'Reilly logo

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Building Microservices with Go

Book Description

Your one-stop guide to the common patterns and practices, showing you how to apply these using the Go programming language

About This Book

  • This short, concise, and practical guide is packed with real-world examples of building microservices with Go
  • It is easy to read and will benefit smaller teams who want to extend the functionality of their existing systems
  • Using this practical approach will save your money in terms of maintaining a monolithic architecture and demonstrate capabilities in ease of use

Who This Book Is For

You should have a working knowledge of programming in Go, including writing and compiling basic applications. However, no knowledge of RESTful architecture, microservices, or web services is expected. If you are looking to apply techniques to your own projects, taking your first steps into microservice architecture, this book is for you.

What You Will Learn

  • Plan a microservice architecture and design a microservice
  • Write a microservice with a RESTful API and a database
  • Understand the common idioms and common patterns in microservices architecture
  • Leverage tools and automation that helps microservices become horizontally scalable
  • Get a grounding in containerization with Docker and Docker-Compose, which will greatly accelerate your development lifecycle
  • Manage and secure Microservices at scale with monitoring, logging, service discovery, and automation
  • Test microservices and integrate API tests in Go

In Detail

Microservice architecture is sweeping the world as the de facto pattern to build web-based applications. Golang is a language particularly well suited to building them. Its strong community, encouragement of idiomatic style, and statically-linked binary artifacts make integrating it with other technologies and managing microservices at scale consistent and intuitive. This book will teach you the common patterns and practices, showing you how to apply these using the Go programming language.

It will teach you the fundamental concepts of architectural design and RESTful communication, and show you patterns that provide manageable code that is supportable in development and at scale in production. We will provide you with examples on how to put these concepts and patterns into practice with Go.

Whether you are planning a new application or working in an existing monolith, this book will explain and illustrate with practical examples how teams of all sizes can start solving problems with microservices. It will help you understand Docker and Docker-Compose and how it can be used to isolate microservice dependencies and build environments. We finish off by showing you various techniques to monitor, test, and secure your microservices.

By the end, you will know the benefits of system resilience of a microservice and the advantages of Go stack.

Style and approach

The step-by-step tutorial focuses on building microservices. Each chapter expands upon the previous one, teaching you the main skills and techniques required to be a successful microservice practitioner.

Downloading the example code for this book. You can download the example code files for all Packt books you have purchased from your account at http://www.PacktPub.com. If you purchased this book elsewhere, you can visit http://www.PacktPub.com/support and register to have the code file.

Table of Contents

  1. Title Page
  2. Copyright
    1. Building Microservices with Go
  3. Credits
  4. About the Author
  5. About the Reviewers
  6. www.PacktPub.com
    1. Why subscribe?
  7. Customer Feedback
  8. Preface
    1. What this book covers
    2. What you need for this book
    3. Who this book is for
    4. Conventions
    5. Reader feedback
    6. Customer support
      1. Downloading the example code
      2. Errata
      3. Piracy
      4. Questions
  9. Introduction to Microservices
    1. Building a simple web server with net/http
    2. Reading and writing JSON
      1. Marshalling Go structs to JSON
      2. Unmarshalling JSON to Go structs
    3. Routing in net/http
      1. Paths
      2. Convenience handlers
      3. FileServer
      4. NotFoundHandler
      5. RedirectHandler
      6. StripPrefix
      7. TimeoutHandler
      8. Static file handler
      9. Creating handlers
    4. Context
      1. Background
      2. WithCancel
      3. WithDeadline
      4. WithTimeout
      5. WithValue
      6. Using contexts
    5. RPC in the Go standard library
      1. Simple RPC example
      2. RPC over HTTP
      3. JSON-RPC over HTTP
    6. Summary
  10. Designing a Great API
    1. RESTful APIs
      1. URIs
      2. URI format
      3. URI path design for REST services
        1. Collections
        2. Documents
        3. Controller
        4. Store
        5. CRUD function names
      4. HTTP verbs
        1. GET
        2. POST
        3. PUT
        4. PATCH
        5. DELETE
        6. HEAD
        7. OPTIONS
      5. URI query design
      6. Response codes
        1. 2xx Success
          1. 200 OK
          2. 201 Created
          3. 204 No Content
        2. 3xx Redirection
          1. 301 Moved Permanently
          2. 304 Not Modified
        3. 4xx Client Error
          1. 400 Bad Request
          2. 401 Unauthorized
          3. 403 Forbidden
          4. 404 Not Found
          5. 405 Method Not Allowed
          6. 408 Request Timeout
        4. 5xx Server Error
          1. 500 Internal Server Error
          2. 503 Service Unavailable
      7. HTTP headers
        1. Standard request headers
          1. Authorization - string
          2. Date
          3. Accept - content type
          4. Accept-Encoding - gzip, deflate
        2. Standard response headers
        3. Returning errors
      8. Accessing APIs from JavaScript
        1. JSONP
        2. CORS
    2. RPC APIs
      1. RPC API design
        1. RPC message frameworks
          1. Gob
          2. Thrift
          3. Protocol Buffers
          4. JSON-RPC
        2. Filtering
    3. Versioning APIs
      1. Semantic versioning
      2. Versioning formats for REST APIs
      3. Versioning formats for RPC APIs
        1. Naming for RPC
    4. Object type standardization
      1. Dates
        1. Durations
        2. Intervals
    5. Documenting APIs
      1. REST based-based APIs
        1. Swagger
        2. API Blueprint
        3. RAML
      2. RPC based-based APIs
    6. Summary
  11. Introducing Docker
    1. Introducing Containers with Docker
    2. Installing Docker
    3. Running our first container
    4. Docker volumes
      1. Union filesystem
      2. Mounting volumes
    5. Docker ports
    6. Removing a container starting with an explicit name
    7. Docker networking
      1. Bridge networking
      2. Host networking
      3. No network
      4. Overlay network
      5. Custom network drivers
        1. Weaveworks
        2. Project Calico
      6. Creating custom bridge networks
        1. Creating a bridge network
          1. Connecting containers to a custom network
    8. Writing Dockerfiles
      1. Building application code for Docker
        1. FROM
        2. MAINTAINER
        3. EXPOSE
        4. COPY
        5. ENTRYPOINT
        6. CMD
        7. Good practice for creating Dockerfiles
      2. Building images from Dockerfiles
        1. Docker build context
        2. Docker Ignore files
      3. Running Daemons in containers
    9. Docker Compose
      1. Installing Docker Compose on Linux
      2. Service startup
      3. Specifying the location of a compose file
      4. Specifying a project name
    10. Summary
  12. Testing
    1. The testing pyramid
    2. Outside-in development
      1. Unit tests
        1. httptest.NewRequest
        2. httptest.NewRecorder
      2. Dependency injection and mocking
      3. Code coverage
      4. Behavioral Driven Development
      5. Testing with Docker Compose
    3. Benchmarking and profiling
      1. Benchmarks
      2. Profiling
    4. Summary
  13. Common Patterns
    1. Design for failure
    2. Patterns
      1. Event processing
        1. Event processing with at least once delivery
          1. Handling Errors
          2. Dead Letter Queue
        2. Idempotent transactions and message order
        3. Atomic transactions
      2. Timeouts
      3. Back off
      4. Circuit breaking
      5. Health checks
      6. Throttling
      7. Service discovery
        1. Server-side service discovery
        2. Client-side service discovery
      8. Load balancing
      9. Caching
        1. Premature optimization
        2. Stale cache in times of database or downstream service failure
    3. Summary
  14. Microservice Frameworks
    1. What makes a good microservice framework?
    2. Micro
      1. Setup
      2. Code generation
      3. Tooling (CI/CD, cross platform)
      4. Maintainable
      5. Format (REST/RPC)
      6. Patterns
      7. Language independence
      8. Ability to interface with other frameworks
      9. Efficiency
      10. Quality
      11. Open source
      12. Security
      13. Support
      14. Extensibility
      15. What we learned about Micro
    3. Kite
      1. Setup
      2. Code generation
      3. Tooling
      4. Maintainable
      5. Format
      6. Patterns
      7. Language independence
      8. Efficiency
      9. Quality
      10. Open source
      11. Security
      12. Support
      13. Extensibility
      14. Summing up Kite
    4. gRPC
      1. Setup
      2. Code generation
      3. Tooling
      4. Maintainable
      5. Format
      6. Patterns
      7. Language independence
      8. Efficiency
      9. Quality
      10. Open source
      11. Security
      12. Support
      13. Extensibility
      14. A few lines about gRPC
    5. Summary
  15. Logging and Monitoring
    1. Logging best practices
    2. Metrics
      1. Types of data best represented by metrics
      2. Naming conventions
      3. Storage and querying
        1. Software as a Service
        2. Self-hosted
      4. Grafana
    3. Logging
      1. Distributed tracing with Correlation IDs
      2. Elasticsearch, Logstash, and Kibana (ELK)
      3. Kibana
    4. Exceptions
      1. Panic and recover
        1. Panic
        2. Recover
    5. Summary
  16. Security
    1. Encryption and signing
      1. Symmetric-key encryption
      2. Public-key cryptography
        1. Digital signatures
      3. X.509 digital certificates
      4. TLS/SSL
    2. External security
      1. Layer 2 or 3 firewalls
      2. Web application firewall
      3. API Gateway
      4. DDoS protection
        1. Types of DDoS attack
          1. UDP fragment attack
          2. UDP flood
          3. DNS
          4. NTP
          5. CHARGEN
          6. SYN flood
          7. SSDP
          8. ACK
    3. Application security
      1. Prevention
      2. Detection
      3. Response
      4. Recovery
      5. Confused deputy
      6. How an attacker could bypass the firewall
        1. Scenario
          1. Attack
      7. Input validation
        1. Fuzzing
      8. TLS
        1. Generating private keys
        2. Generating X.509 certificates
      9. Securing data at rest
        1. Physical machine access
      10. OWASP
        1. Never storing session tokens in a URL
        2. Cross-site scripting (XSS) and cross-site request forgery (CRSF)
        3. Insecure direct object references
      11. Authentication and authorization
      12. Password hashing
        1. Adding a little seasoning
        2. Dictionary attacks
        3. Adding a pepper
        4. bcrypt
        5. Benchmarks
      13. JWTs
        1. Format of a JWT
      14. Secure messages
      15. Shared secrets
      16. Asymmetric encryption with large messages
    4. Maintenance
      1. Patching containers
      2. Software updates
      3. Patching application code
      4. Logging
    5. Summary
  17. Event-Driven Architecture
    1. Differences between synchronous and asynchronous processing
      1. Synchronous processing
      2. Asynchronous processing
    2. Types of asynchronous messages
      1. Pull/queue messaging
      2. Push messaging
    3. Command Query Responsibility Segregation (CQRS)
    4. Domain-driven design
      1. What is DDD?
      2. Technical debt
      3. Anatomy of DDD
        1. Strategic design
        2. Tactical design
        3. Ubiquitous language
        4. Bounded contexts
        5. Context mapping
    5. Software
      1. Kafka
      2. NATS.io
      3. AWS SNS/SQS
      4. Google Cloud Pub/Sub
    6. Summary
  18. Continuous Delivery
    1. What is continuous delivery?
      1. Manual deployment
      2. The benefits of continuous delivery
      3. Aspects of continuous delivery
        1. Reproducibility and consistency
        2. Artifact storage
        3. Automation of tests
        4. Automation of integration tests
        5. Infrastructure as code
        6. Security scanning
        7. Static code analysis
        8. Smoke tests
        9. End-to-end tests
        10. Monitoring
      4. Continuous delivery process
      5. Overview
    2. What is container orchestration?
      1. Options for container orchestration
    3. What is immutable infrastructure?
    4. Terraform
      1. Providers
      2. Terraform config entry point
      3. VPC module
      4. Output variables
      5. Creating the infrastructure
    5. Example application
      1. Continuous delivery workflow
        1. Build
        2. Testing
        3. Benchmarking
        4. Static code analysis
        5. Integration tests
        6. Deployment
        7. Smoke tests
        8. Monitoring/alerting
        9. Complete workflow
    6. Summary