Concurrency in Go

Chapter 5. Concurrency at Scale

Now that you’ve learned some common patterns for utilizing concurrency within Go, let’s turn our attention to composing these patterns into a series of practices that will enable you to write large, composable systems that scale.

In this chapter, we’ll discuss ways to scale concurrent operations within a single process, and also begin looking at how concurrency comes into play when dealing with more than one process.

Error Propagation

With concurrent code, and especially distributed systems, it’s both easy for something to go wrong in your system, and difficult to understand why it happened. You can save yourself, your team, and your users a whole lot of pain by carefully considering how issues propagate through your system, and how they end up being represented to the user. In the section “Error Handling”, we discussed how to propagate errors from goroutines, but we didn’t spend any time discussing what those errors should look like, or how errors should flow through a large and complex system. Let’s spend some time here discussing a philosophy of error propagation. What follows is an opinionated framework for handling errors in concurrent systems.

Many developers make the mistake of thinking of error propagation as secondary, or “other,” to the flow of their system. Careful consideration is given to how data flows through the system, but errors are something that are tolerated and ferried up the stack without much thought, and ultimately dumped ...