Efficient Node.js

Chapter 6. Streams

A Note for Early Release Readers

With Early Release ebooks, you get books in their earliest form—the author’s raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles.

This will be the sixth chapter of the final book. Please note that the GitHub repo will be made active later on.

If you have comments about how we might improve the content and/or examples in this book, or if you notice missing material within this chapter, please reach out to the author at [e-mail].

Node streams have a reputation for being hard to understand and work with. That might have been the case in the early days of Node, but things have changed. Today, it’s relatively easy to create and consume streams in Node. We can even use native JavaScript async iterators and generators to work with Node streams.

In this chapter, I’ll explain the concept of streams, why they are needed, how to create them, use them, and combine them to efficiently process large amounts of data without overwhelming the memory available to a Node process.

Introduction

When you download a file from the internet, watch a show, or listen to a song, you are using streams. The content is being streamed to you one chunk at a time.

Streams are basically collections of data, similar to arrays or strings, but instead of storing data in memory space, streams process data over time. You can use streams to process very large amounts ...