book

Efficient Go

by Bartlomiej Plotka

November 2022

Intermediate to advanced

502 pages

13h 50m

English

O'Reilly Media, Inc.

Book available

Read now

Unlock full access

Why I Wrote This BookHow I Gathered This KnowledgeWho This Book Is ForHow This Book Is OrganizedConventions Used in This BookUsing Code ExamplesAcknowledgmentsFeedback Is Welcome!O’Reilly Online LearningHow to Contact Us
Behind PerformanceCommon Efficiency MisconceptionsOptimized Code Is Not ReadableYou Aren’t Going to Need ItHardware Is Getting Faster and CheaperWe Can Scale Horizontally InsteadTime to Market Is More ImportantThe Key to Pragmatic Code PerformanceSummary
Basics You Should Know About GoImperative, Compiled, and Statically Typed LanguageDesigned to Improve Serious CodebasesGoverned by Google, Yet Open SourceSimplicity, Safety, and Readability Are ParamountPackaging and ModulesDependencies Transparency by DefaultConsistent ToolingSingle Way of Handling ErrorsStrong EcosystemUnused Import or Variable Causes Build ErrorUnit Testing and Table TestsAdvanced Language ElementsCode Documentation as a First CitizenBackward Compatibility and PortabilityGo RuntimeObject-Oriented ProgrammingGenericsIs Go “Fast”?Summary
Beyond Waste, Optimization Is a Zero-Sum GameReasonable OptimizationsDeliberate OptimizationsOptimization ChallengesUnderstand Your GoalsEfficiency Requirements Should Be FormalizedResource-Aware Efficiency RequirementsAcquiring and Assessing Efficiency GoalsExample of Defining RAERGot an Efficiency Problem? Keep Calm!Optimization Design LevelsEfficiency-Aware Development FlowFunctionality PhaseEfficiency PhaseSummary
CPU in a Modern Computer ArchitectureAssemblyUnderstanding Go CompilerCPU and Memory Wall ProblemHierachical Cache SystemPipelining and Out-of-Order ExecutionHyper-ThreadingSchedulersOperating System SchedulerGo Runtime SchedulerWhen to Use ConcurrencySummary
Memory RelevanceDo We Have a Memory Problem?Physical MemoryOS Memory ManagementVirtual Memorymmap SyscallOS Memory MappingGo Memory ManagementValues, Pointers, and Memory BlocksGo AllocatorGarbage CollectionSummary
ObservabilityExample: Instrumenting for LatencyLoggingTracingMetricsEfficiency Metrics SemanticsLatencyCPU UsageMemory UsageSummary
Complexity Analysis“Estimated” Efficiency ComplexityAsymptotic Complexity with Big O NotationPractical ApplicationsThe Art of BenchmarkingComparison to Functional TestingBenchmarks LieReliability of ExperimentsHuman ErrorsReproducing ProductionPerformance NondeterminismBenchmarking LevelsBenchmarking in ProductionMacrobenchmarksMicrobenchmarksWhat Level Should You Use?Summary
MicrobenchmarksGo BenchmarksUnderstanding the ResultsTips and Tricks for MicrobenchmarkingToo-High VarianceFind Your WorkflowTest Your Benchmark for Correctness!Sharing Benchmarks with the Team (and Your Future Self)Running Benchmarks for Different InputsMicrobenchmarks Versus Memory ManagementCompiler Optimizations Versus BenchmarkMacrobenchmarksBasicsGo e2e FrameworkUnderstanding Results and ObservationsCommon Macrobenchmarking WorkflowsSummary
Root Cause Analysis, but for EfficiencyProfiling in Gopprof Formatgo tool pprof ReportsCapturing the Profiling SignalCommon Profile InstrumentationHeapGoroutineCPUOff-CPU TimeTips and TricksSharing ProfilesContinuous ProfilingComparing and Aggregating ProfilesSummary

Sum ExamplesOptimizing LatencyOptimizing bytes.SplitOptimizing runtime.slicebytetostringOptimizing strconv.ParseOptimizing Memory UsageMoving to Streaming AlgorithmOptimizing bufio.ScannerOptimizing Latency Using ConcurrencyA Naive ConcurrencyA Worker Approach with DistributionA Worker Approach Without Coordination (Sharding)A Streamed, Sharded Worker ApproachBonus: Thinking Out of the BoxSummary
Common PatternsDo Less WorkTrading Functionality for EfficiencyTrading Space for TimeTrading Time for SpaceThe Three Rs Optimization MethodReduce AllocationsReuse MemoryRecycleDon’t Leak ResourcesControl the Lifecycle of Your GoroutinesReliably Close ThingsExhaust ThingsPre-Allocate If You CanOverusing Memory with ArraysMemory Reuse and PoolingSummaryNext Steps

Content preview from Efficient Go

Chapter 2. Efficient Introduction to Go

Go is efficient, scalable, and productive. Some programmers find it fun to work in; others find it unimaginative, even boring. ... Those are not contradictory positions. Go was designed to address the problems faced in software development at Google, which led to a language that is not a breakthrough research language but is nonetheless an excellent tool for engineering large software projects.

Rob Pike, “Go at Google: Language Design in the Service of Software Engineering”

I am a huge fan of the Go programming language. The number of things developers around the world have been able to achieve with Go is impressive. For a few years in a row, Go has been on the list of top five languages people love or want to learn. It is used in many businesses, including bigger tech companies like Apple, American Express, Cloudflare, Dell, Google, Netflix, Red Hat, Twitch, and others. Of course, as with everything, nothing is perfect. I would probably change, remove, or add a few things to Go, but if you would wake me in the middle of the night and ask me to quickly write reliable backend code, I would write it in Go. CLI? In Go. Quick, reliable script? In Go as well. The first language to learn as a junior programmer? Go. Code for IoT, robots, and microprocessors? The answer is also Go.¹ Infrastructure configuration? As of 2022, I don’t think there is a better tool for robust templating than Go.²

Don’t get me wrong, there are languages with specialized ...