Efficient Go

Book description

With technological advancements, fast markets, and higher complexity of systems, software engineers tend to skip the uncomfortable topic of software efficiency. However, tactical, observability-driven performance optimizations are vital for every product to save money and ensure business success.

With this book, any engineer can learn how to approach software efficiency effectively, professionally, and without stress. Author Bartłomiej Płotka provides the tools and knowledge required to make your systems faster and less resource-hungry. Efficient Go guides you in achieving better day-to-day efficiency using Go. In addition, most content is language-agnostic, allowing you to bring small but effective habits to your programming or product management cycles.

This book shows you how to:

  • Clarify and negotiate efficiency goals
  • Optimize efficiency on various levels
  • Use common resources like CPU and memory effectively
  • Assess efficiency using observability signals like metrics, logging, tracing, and (continuous) profiling via open source projects like Prometheus, Jaeger, and Parca
  • Apply tools like go test , pprof , benchstat , and k6 to create reliable micro and macro benchmarks
  • Efficiently use Go and its features like slices, generics, goroutines, allocation semantics, garbage collection, and more!

Table of contents

  1. Preface
    1. Why I Wrote This Book
    2. How I Gathered This Knowledge
    3. Who This Book Is For
    4. How This Book Is Organized
    5. Conventions Used in This Book
    6. Using Code Examples
    7. Acknowledgments
    8. Feedback Is Welcome!
    9. O’Reilly Online Learning
    10. How to Contact Us
  2. 1. Software Efficiency Matters
    1. Behind Performance
    2. Common Efficiency Misconceptions
      1. Optimized Code Is Not Readable
      2. You Aren’t Going to Need It
      3. Hardware Is Getting Faster and Cheaper
      4. We Can Scale Horizontally Instead
      5. Time to Market Is More Important
    3. The Key to Pragmatic Code Performance
    4. Summary
  3. 2. Efficient Introduction to Go
    1. Basics You Should Know About Go
      1. Imperative, Compiled, and Statically Typed Language
      2. Designed to Improve Serious Codebases
      3. Governed by Google, Yet Open Source
      4. Simplicity, Safety, and Readability Are Paramount
      5. Packaging and Modules
      6. Dependencies Transparency by Default
      7. Consistent Tooling
      8. Single Way of Handling Errors
      9. Strong Ecosystem
      10. Unused Import or Variable Causes Build Error
      11. Unit Testing and Table Tests
    2. Advanced Language Elements
      1. Code Documentation as a First Citizen
      2. Backward Compatibility and Portability
      3. Go Runtime
      4. Object-Oriented Programming
      5. Generics
    3. Is Go “Fast”?
    4. Summary
  4. 3. Conquering Efficiency
    1. Beyond Waste, Optimization Is a Zero-Sum Game
      1. Reasonable Optimizations
      2. Deliberate Optimizations
    2. Optimization Challenges
    3. Understand Your Goals
      1. Efficiency Requirements Should Be Formalized
      2. Resource-Aware Efficiency Requirements
      3. Acquiring and Assessing Efficiency Goals
      4. Example of Defining RAER
    4. Got an Efficiency Problem? Keep Calm!
    5. Optimization Design Levels
    6. Efficiency-Aware Development Flow
      1. Functionality Phase
      2. Efficiency Phase
    7. Summary
  5. 4. How Go Uses the CPU Resource (or Two)
    1. CPU in a Modern Computer Architecture
    2. Assembly
    3. Understanding Go Compiler
    4. CPU and Memory Wall Problem
      1. Hierachical Cache System
      2. Pipelining and Out-of-Order Execution
      3. Hyper-Threading
    5. Schedulers
      1. Operating System Scheduler
      2. Go Runtime Scheduler
    6. When to Use Concurrency
    7. Summary
  6. 5. How Go Uses Memory Resource
    1. Memory Relevance
    2. Do We Have a Memory Problem?
    3. Physical Memory
    4. OS Memory Management
      1. Virtual Memory
      2. mmap Syscall
      3. OS Memory Mapping
    5. Go Memory Management
      1. Values, Pointers, and Memory Blocks
      2. Go Allocator
      3. Garbage Collection
    6. Summary
  7. 6. Efficiency Observability
    1. Observability
    2. Example: Instrumenting for Latency
      1. Logging
      2. Tracing
      3. Metrics
    3. Efficiency Metrics Semantics
      1. Latency
      2. CPU Usage
      3. Memory Usage
    4. Summary
  8. 7. Data-Driven Efficiency Assessment
    1. Complexity Analysis
      1. “Estimated” Efficiency Complexity
      2. Asymptotic Complexity with Big O Notation
      3. Practical Applications
    2. The Art of Benchmarking
      1. Comparison to Functional Testing
      2. Benchmarks Lie
    3. Reliability of Experiments
      1. Human Errors
      2. Reproducing Production
      3. Performance Nondeterminism
    4. Benchmarking Levels
      1. Benchmarking in Production
      2. Macrobenchmarks
      3. Microbenchmarks
      4. What Level Should You Use?
    5. Summary
  9. 8. Benchmarking
    1. Microbenchmarks
      1. Go Benchmarks
      2. Understanding the Results
    2. Tips and Tricks for Microbenchmarking
      1. Too-High Variance
      2. Find Your Workflow
      3. Test Your Benchmark for Correctness!
      4. Sharing Benchmarks with the Team (and Your Future Self)
      5. Running Benchmarks for Different Inputs
      6. Microbenchmarks Versus Memory Management
      7. Compiler Optimizations Versus Benchmark
    3. Macrobenchmarks
      1. Basics
      2. Go e2e Framework
      3. Understanding Results and Observations
    4. Common Macrobenchmarking Workflows
    5. Summary
  10. 9. Data-Driven Bottleneck Analysis
    1. Root Cause Analysis, but for Efficiency
    2. Profiling in Go
      1. pprof Format
      2. go tool pprof Reports
    3. Capturing the Profiling Signal
    4. Common Profile Instrumentation
      1. Heap
      2. Goroutine
      3. CPU
      4. Off-CPU Time
    5. Tips and Tricks
      1. Sharing Profiles
      2. Continuous Profiling
      3. Comparing and Aggregating Profiles
    6. Summary
  11. 10. Optimization Examples
    1. Sum Examples
    2. Optimizing Latency
      1. Optimizing bytes.Split
      2. Optimizing runtime.slicebytetostring
      3. Optimizing strconv.Parse
    3. Optimizing Memory Usage
      1. Moving to Streaming Algorithm
      2. Optimizing bufio.Scanner
    4. Optimizing Latency Using Concurrency
      1. A Naive Concurrency
      2. A Worker Approach with Distribution
      3. A Worker Approach Without Coordination (Sharding)
      4. A Streamed, Sharded Worker Approach
    5. Bonus: Thinking Out of the Box
    6. Summary
  12. 11. Optimization Patterns
    1. Common Patterns
      1. Do Less Work
      2. Trading Functionality for Efficiency
      3. Trading Space for Time
      4. Trading Time for Space
    2. The Three Rs Optimization Method
      1. Reduce Allocations
      2. Reuse Memory
      3. Recycle
    3. Don’t Leak Resources
      1. Control the Lifecycle of Your Goroutines
      2. Reliably Close Things
      3. Exhaust Things
    4. Pre-Allocate If You Can
    5. Overusing Memory with Arrays
    6. Memory Reuse and Pooling
    7. Summary
    8. Next Steps
  13. A. Latencies for Napkin Math Calculations
  14. Index
  15. About the Author

Product information

  • Title: Efficient Go
  • Author(s): Bartlomiej Plotka
  • Release date: November 2022
  • Publisher(s): O'Reilly Media, Inc.
  • ISBN: 9781098105716