book

Cloud Native Go

by Matthew A. Titmus

April 2021

Intermediate to advanced

433 pages

10h 45m

English

O'Reilly Media, Inc.

Book available

Read now

Unlock full access

Who Should Read This BookWhy I Wrote This BookConventions Used in This BookUsing Code ExamplesO’Reilly Online LearningHow to Contact UsAcknowledgments
The Story So FarWhat Is Cloud Native?ScalabilityLoose CouplingResilienceManageabilityObservabilityWhy Is Cloud Native a Thing?Summary
The Motivation Behind GoFeatures for a Cloud Native WorldComposition and Structural TypingComprehensibilityCSP-Style ConcurrencyFast BuildsLinguistic StabilityMemory SafetyPerformanceStatic LinkingStatic TypingSummary
Basic Data TypesBooleansSimple NumbersComplex NumbersStringsVariablesShort Variable DeclarationsZero ValuesThe Blank IdentifierConstantsContainer Types: Arrays, Slices, and MapsArraysSlicesMapsPointersControl StructuresFun with forThe if StatementThe switch StatementError HandlingCreating an ErrorPutting the Fun in Functions: Variadics and ClosuresFunctionsVariadic FunctionsAnonymous Functions and ClosuresStructs, Methods, and InterfacesStructsMethodsInterfacesComposition with Type EmbeddingThe Good Stuff: ConcurrencyGoroutinesChannelsSelectSummary
The Context PackageWhat Context Can Do for YouCreating ContextDefining Context Deadlines and TimeoutsDefining Request-Scoped ValuesUsing a ContextLayout of this ChapterStability PatternsCircuit BreakerDebounceRetryThrottleTimeoutConcurrency PatternsFan-InFan-OutFutureShardingSummary
Let’s Build a Service!What’s a Key-Value Store?RequirementsWhat Is Idempotence and Why Does It Matter?The Eventual GoalGeneration 0: The Core FunctionalityYour Super Simple APIGeneration 1: The MonolithBuilding an HTTP Server with net/httpBuilding an HTTP Server with gorilla/muxBuilding a RESTful ServiceMaking Your Data Structure Concurrency-SafeGeneration 2: Persisting Resource StateWhat’s a Transaction Log?Storing State in a Transaction Log FileStoring State in an External DatabaseGeneration 3: Implementing Transport Layer SecurityTransport Layer SecurityPrivate Key and Certificate FilesSecuring Your Web Service with HTTPSTransport Layer SummaryContainerizing Your Key-Value StoreDocker (Absolute) BasicsBuilding Your Key-Value Store ContainerExternalizing Container DataSummary
What’s the Point of Cloud Native?It’s All About DependabilityWhat Is Dependability and Why Is It So Important?Dependability: It’s Not Just for Ops AnymoreAchieving DependabilityFault PreventionFault ToleranceFault RemovalFault ForecastingThe Continuing Relevance of the Twelve-Factor AppI. CodebaseII. DependenciesIII. ConfigurationIV. Backing ServicesV. Build, Release, RunVI. ProcessesVII. Data IsolationVIII. ScalabilityIX. DisposabilityX. Development/Production ParityXI. LogsXII. Administrative ProcessesSummary

What Is Scalability?Different Forms of ScalingThe Four Common BottlenecksState and StatelessnessApplication State Versus Resource StateAdvantages of StatelessnessScaling Postponed: EfficiencyEfficient Caching Using an LRU CacheEfficient SynchronizationMemory Leaks Can…fatal error: runtime: out of memoryOn EfficiencyService ArchitecturesThe Monolith System ArchitectureThe Microservices System ArchitectureServerless ArchitecturesSummary
Tight CouplingTight Coupling Takes Many FormsCommunications Between ServicesRequest-Response MessagingCommon Request-Response ImplementationsIssuing HTTP Requests with net/httpRemote Procedure Calls with gRPCLoose Coupling Local Resources with Plug-insIn-Process Plug-ins with the plugin PackageHashiCorp’s Go Plug-in System over RPCHexagonal ArchitectureThe ArchitectureImplementing a Hexagonal ServiceSummary
Keeping on Ticking: Why Resilience MattersWhat Does It Mean for a System to Fail?Building for ResilienceCascading FailuresPreventing OverloadPlay It Again: Retrying RequestsBackoff AlgorithmsCircuit BreakingTimeoutsIdempotenceService RedundancyDesigning for RedundancyAutoscalingHealthy Health ChecksWhat Does It Mean for an Instance to Be “Healthy”?The Three Types of Health ChecksFailing OpenSummary
What Is Manageability and Why Should I Care?Configuring Your ApplicationConfiguration Good PracticeConfiguring with Environment VariablesConfiguring with Command-Line ArgumentsConfiguring with FilesViper: The Swiss Army Knife of Configuration PackagesFeature Management with Feature FlagsThe Evolution of a Feature FlagGeneration 0: The Initial ImplementationGeneration 1: The Hard-Coded Feature FlagGeneration 2: The Configurable FlagGeneration 3: Dynamic Feature FlagsSummary
What Is Observability?Why Do We Need Observability?How Is Observability Different from “Traditional” Monitoring?The “Three Pillars of Observability”OpenTelemetryThe OpenTelemetry ComponentsTracingTracing ConceptsTracing with OpenTelemetryPutting It All Together: TracingMetricsPush Versus Pull Metric CollectionMetrics with OpenTelemetryPutting It All Together: MetricsLoggingBetter Logging PracticesLogging with Go’s Standard log PackageThe Zap Logging PackageSummary

Content preview from Cloud Native Go

Chapter 9. Resilience

A distributed system is one in which the failure of a computer you didn’t even know about can render your own computer unusable.¹

Leslie Lamport, DEC SRC Bulletin Board (May 1987)

Late one September night, at just after two in the morning, a portion of Amazon’s internal network quietly stopped working.² This event was brief, and not particularly interesting, except that it happened to affect a sizable number of the servers that supported the DynamoDB service.

Most days, this wouldn’t be such a big deal. Any affected servers would just try to reconnect to the cluster by retrieving their membership data from a dedicated metadata service. If that failed, they would temporarily take themselves offline and try again.

But this time, when the network was restored, a small army of storage servers simultaneously requested their membership data from the metadata service, overwhelming it so that requests—even ones from previously unaffected servers—started to time out. Storage servers dutifully responded to the timeouts by taking themselves offline and retrying (again), further stressing the metadata service, causing even more servers to go offline, and so on. Within minutes, the outage had spread to the entire cluster. The service was effectively down, taking a number of dependent services down with it.

To make matters worse, the sheer volume of retry attempts—a “retry storm”—put such a burden on the metadata service that it even became entirely unresponsive to requests ...