Building Microservices with Go

Building Microservices with Go

by Nic Jackson
Released July 2017
Publisher(s): Packt Publishing
ISBN: 9781786468666

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

Your one-stop guide to the common patterns and practices, showing you how to apply these using the Go programming language

About This Book

  • This short, concise, and practical guide is packed with real-world examples of building microservices with Go

  • It is easy to read and will benefit smaller teams who want to extend the functionality of their existing systems

  • Using this practical approach will save your money in terms of maintaining a monolithic architecture and demonstrate capabilities in ease of use

    • Who This Book Is For

    You should have a working knowledge of programming in Go, including writing and compiling basic applications. However, no knowledge of RESTful architecture, microservices, or web services is expected. If you are looking to apply techniques to your own projects, taking your first steps into microservice architecture, this book is for you.

    What You Will Learn

  • Plan a microservice architecture and design a microservice

  • Write a microservice with a RESTful API and a database

  • Understand the common idioms and common patterns in microservices architecture

  • Leverage tools and automation that helps microservices become horizontally scalable

  • Get a grounding in containerization with Docker and Docker-Compose, which will greatly accelerate your development lifecycle

  • Manage and secure Microservices at scale with monitoring, logging, service discovery, and automation

  • Test microservices and integrate API tests in Go

    • In Detail

    Microservice architecture is sweeping the world as the de facto pattern to build web-based applications. Golang is a language particularly well suited to building them. Its strong community, encouragement of idiomatic style, and statically-linked binary artifacts make integrating it with other technologies and managing microservices at scale consistent and intuitive. This book will teach you the common patterns and practices, showing you how to apply these using the Go programming language.

    It will teach you the fundamental concepts of architectural design and RESTful communication, and show you patterns that provide manageable code that is supportable in development and at scale in production. We will provide you with examples on how to put these concepts and patterns into practice with Go.

    Whether you are planning a new application or working in an existing monolith, this book will explain and illustrate with practical examples how teams of all sizes can start solving problems with microservices. It will help you understand Docker and Docker-Compose and how it can be used to isolate microservice dependencies and build environments. We finish off by showing you various techniques to monitor, test, and secure your microservices.

    By the end, you will know the benefits of system resilience of a microservice and the advantages of Go stack.

    Style and approach

    The step-by-step tutorial focuses on building microservices. Each chapter expands upon the previous one, teaching you the main skills and techniques required to be a successful microservice practitioner.

    Table of contents

    1. Title Page
    2. Copyright
      1. Building Microservices with Go
    3. Credits
    4. About the Author
    5. About the Reviewers
    6. www.PacktPub.com
      1. Why subscribe?
    7. Customer Feedback
    8. Preface
      1. What this book covers
      2. What you need for this book
      3. Who this book is for
      4. Conventions
      5. Reader feedback
      6. Customer support
        1. Downloading the example code
        2. Errata
        3. Piracy
        4. Questions
    9. Introduction to Microservices
      1. Building a simple web server with net/http
      2. Reading and writing JSON
        1. Marshalling Go structs to JSON
        2. Unmarshalling JSON to Go structs
      3. Routing in net/http
        1. Paths
        2. Convenience handlers
        3. FileServer
        4. NotFoundHandler
        5. RedirectHandler
        6. StripPrefix
        7. TimeoutHandler
        8. Static file handler
        9. Creating handlers
      4. Context
        1. Background
        2. WithCancel
        3. WithDeadline
        4. WithTimeout
        5. WithValue
        6. Using contexts
      5. RPC in the Go standard library
        1. Simple RPC example
        2. RPC over HTTP
        3. JSON-RPC over HTTP
      6. Summary
    10. Designing a Great API
      1. RESTful APIs
        1. URIs
        2. URI format
        3. URI path design for REST services
          1. Collections
          2. Documents
          3. Controller
          4. Store
          5. CRUD function names
        4. HTTP verbs
          1. GET
          2. POST
          3. PUT
          4. PATCH
          5. DELETE
          6. HEAD
          7. OPTIONS
        5. URI query design
        6. Response codes
          1. 2xx Success
            1. 200 OK
            2. 201 Created
            3. 204 No Content
          2. 3xx Redirection
            1. 301 Moved Permanently
            2. 304 Not Modified
          3. 4xx Client Error
            1. 400 Bad Request
            2. 401 Unauthorized
            3. 403 Forbidden
            4. 404 Not Found
            5. 405 Method Not Allowed
            6. 408 Request Timeout
          4. 5xx Server Error
            1. 500 Internal Server Error
            2. 503 Service Unavailable
        7. HTTP headers
          1. Standard request headers
            1. Authorization - string
            2. Date
            3. Accept - content type
            4. Accept-Encoding - gzip, deflate
          2. Standard response headers
          3. Returning errors
        8. Accessing APIs from JavaScript
          1. JSONP
          2. CORS
      2. RPC APIs
        1. RPC API design
          1. RPC message frameworks
            1. Gob
            2. Thrift
            3. Protocol Buffers
            4. JSON-RPC
          2. Filtering
      3. Versioning APIs
        1. Semantic versioning
        2. Versioning formats for REST APIs
        3. Versioning formats for RPC APIs
          1. Naming for RPC
      4. Object type standardization
        1. Dates
          1. Durations
          2. Intervals
      5. Documenting APIs
        1. REST based-based APIs
          1. Swagger
          2. API Blueprint
          3. RAML
        2. RPC based-based APIs
      6. Summary
    11. Introducing Docker
      1. Introducing Containers with Docker
      2. Installing Docker
      3. Running our first container
      4. Docker volumes
        1. Union filesystem
        2. Mounting volumes
      5. Docker ports
      6. Removing a container starting with an explicit name
      7. Docker networking
        1. Bridge networking
        2. Host networking
        3. No network
        4. Overlay network
        5. Custom network drivers
          1. Weaveworks
          2. Project Calico
        6. Creating custom bridge networks
          1. Creating a bridge network
            1. Connecting containers to a custom network
      8. Writing Dockerfiles
        1. Building application code for Docker
          1. FROM
          2. MAINTAINER
          3. EXPOSE
          4. COPY
          5. ENTRYPOINT
          6. CMD
          7. Good practice for creating Dockerfiles
        2. Building images from Dockerfiles
          1. Docker build context
          2. Docker Ignore files
        3. Running Daemons in containers
      9. Docker Compose
        1. Installing Docker Compose on Linux
        2. Service startup
        3. Specifying the location of a compose file
        4. Specifying a project name
      10. Summary
    12. Testing
      1. The testing pyramid
      2. Outside-in development
        1. Unit tests
          1. httptest.NewRequest
          2. httptest.NewRecorder
        2. Dependency injection and mocking
        3. Code coverage
        4. Behavioral Driven Development
        5. Testing with Docker Compose
      3. Benchmarking and profiling
        1. Benchmarks
        2. Profiling
      4. Summary
    13. Common Patterns
      1. Design for failure
      2. Patterns
        1. Event processing
          1. Event processing with at least once delivery
            1. Handling Errors
            2. Dead Letter Queue
          2. Idempotent transactions and message order
          3. Atomic transactions
        2. Timeouts
        3. Back off
        4. Circuit breaking
        5. Health checks
        6. Throttling
        7. Service discovery
          1. Server-side service discovery
          2. Client-side service discovery
        8. Load balancing
        9. Caching
          1. Premature optimization
          2. Stale cache in times of database or downstream service failure
      3. Summary
    14. Microservice Frameworks
      1. What makes a good microservice framework?
      2. Micro
        1. Setup
        2. Code generation
        3. Tooling (CI/CD, cross platform)
        4. Maintainable
        5. Format (REST/RPC)
        6. Patterns
        7. Language independence
        8. Ability to interface with other frameworks
        9. Efficiency
        10. Quality
        11. Open source
        12. Security
        13. Support
        14. Extensibility
        15. What we learned about Micro
      3. Kite
        1. Setup
        2. Code generation
        3. Tooling
        4. Maintainable
        5. Format
        6. Patterns
        7. Language independence
        8. Efficiency
        9. Quality
        10. Open source
        11. Security
        12. Support
        13. Extensibility
        14. Summing up Kite
      4. gRPC
        1. Setup
        2. Code generation
        3. Tooling
        4. Maintainable
        5. Format
        6. Patterns
        7. Language independence
        8. Efficiency
        9. Quality
        10. Open source
        11. Security
        12. Support
        13. Extensibility
        14. A few lines about gRPC
      5. Summary
    15. Logging and Monitoring
      1. Logging best practices
      2. Metrics
        1. Types of data best represented by metrics
        2. Naming conventions
        3. Storage and querying
          1. Software as a Service
          2. Self-hosted
        4. Grafana
      3. Logging
        1. Distributed tracing with Correlation IDs
        2. Elasticsearch, Logstash, and Kibana (ELK)
        3. Kibana
      4. Exceptions
        1. Panic and recover
          1. Panic
          2. Recover
      5. Summary
    16. Security
      1. Encryption and signing
        1. Symmetric-key encryption
        2. Public-key cryptography
          1. Digital signatures
        3. X.509 digital certificates
        4. TLS/SSL
      2. External security
        1. Layer 2 or 3 firewalls
        2. Web application firewall
        3. API Gateway
        4. DDoS protection
          1. Types of DDoS attack
            1. UDP fragment attack
            2. UDP flood
            3. DNS
            4. NTP
            5. CHARGEN
            6. SYN flood
            7. SSDP
            8. ACK
      3. Application security
        1. Prevention
        2. Detection
        3. Response
        4. Recovery
        5. Confused deputy
        6. How an attacker could bypass the firewall
          1. Scenario
            1. Attack
        7. Input validation
          1. Fuzzing
        8. TLS
          1. Generating private keys
          2. Generating X.509 certificates
        9. Securing data at rest
          1. Physical machine access
        10. OWASP
          1. Never storing session tokens in a URL
          2. Cross-site scripting (XSS) and cross-site request forgery (CRSF)
          3. Insecure direct object references
        11. Authentication and authorization
        12. Password hashing
          1. Adding a little seasoning
          2. Dictionary attacks
          3. Adding a pepper
          4. bcrypt
          5. Benchmarks
        13. JWTs
          1. Format of a JWT
        14. Secure messages
        15. Shared secrets
        16. Asymmetric encryption with large messages
      4. Maintenance
        1. Patching containers
        2. Software updates
        3. Patching application code
        4. Logging
      5. Summary
    17. Event-Driven Architecture
      1. Differences between synchronous and asynchronous processing
        1. Synchronous processing
        2. Asynchronous processing
      2. Types of asynchronous messages
        1. Pull/queue messaging
        2. Push messaging
      3. Command Query Responsibility Segregation (CQRS)
      4. Domain-driven design
        1. What is DDD?
        2. Technical debt
        3. Anatomy of DDD
          1. Strategic design
          2. Tactical design
          3. Ubiquitous language
          4. Bounded contexts
          5. Context mapping
      5. Software
        1. Kafka
        2. NATS.io
        3. AWS SNS/SQS
        4. Google Cloud Pub/Sub
      6. Summary
    18. Continuous Delivery
      1. What is continuous delivery?
        1. Manual deployment
        2. The benefits of continuous delivery
        3. Aspects of continuous delivery
          1. Reproducibility and consistency
          2. Artifact storage
          3. Automation of tests
          4. Automation of integration tests
          5. Infrastructure as code
          6. Security scanning
          7. Static code analysis
          8. Smoke tests
          9. End-to-end tests
          10. Monitoring
        4. Continuous delivery process
        5. Overview
      2. What is container orchestration?
        1. Options for container orchestration
      3. What is immutable infrastructure?
      4. Terraform
        1. Providers
        2. Terraform config entry point
        3. VPC module
        4. Output variables
        5. Creating the infrastructure
      5. Example application
        1. Continuous delivery workflow
          1. Build
          2. Testing
          3. Benchmarking
          4. Static code analysis
          5. Integration tests
          6. Deployment
          7. Smoke tests
          8. Monitoring/alerting
          9. Complete workflow
      6. Summary

    Product information

    • Title: Building Microservices with Go
    • Author(s): Nic Jackson
    • Release date: July 2017
    • Publisher(s): Packt Publishing
    • ISBN: 9781786468666