Chapter 6. Streams
Node streams have a reputation for being hard to understand and work with. That might have been the case in the early days of Node, but things have changed. Today, it’s relatively easy to create and consume streams in Node. We can even use native JavaScript async iterators and generators to work with Node streams.
In this chapter, I’ll explain the concept of streams, why they’re needed, and how to create, use, and combine them to efficiently process large amounts of data without overwhelming the memory available to a Node process.
Introducing Streams
When you download a file from the internet, watch a show, or listen to a song, you are using streams. The content is being streamed to you one chunk at a time.
Streams are basically collections of data, similar to arrays or strings, but instead of storing data in memory space, streams process data over time. You can use streams to process very large amounts of data using a limited memory space.
Analogies for streams are all around us in life. When you have a sink full of dirty dishes, you can load them up in a dishwasher, and that would be processing all of them at once, like holding them in an array. If you can’t use a dishwasher, you’ll need to employ the concept of streams and clean up the sink one dish at a time. You take one dish, scrub it, rinse it, and then move on to the next one.
This is great if you have a small number of dishes to wash. If you have a lot of dishes to wash and someone who can help, you ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access