Design It!

Pipe-and-Filter Pattern

With the pipe-and-filter pattern, each component called a filter is responsible for a single transformation or data operation. Data is streamed from one filter to the next as quickly as possible, and data operations occur in parallel. Loosely coupled filters can be reused and combined in different ways to create new pipelines.

The pipe-and-filter pattern is prevalent in data analysis and data transformation use cases. If you’ve ever piped Unix commands together in a terminal window, then you have firsthand experience with the pipe-and-filter pattern. See the table.

Category	Component & Connector
Elements	Filter—A component that reads data, transforms it, then writes out the transformed data. Filters may begin processing ...