Software Architect's Handbook

Logstash

Logstash is an open-source log-parsing engine that provides functionality to parse, transform, and transport data. It aggregates, filters, and supplements data from a variety of sources. One of the many uses to which it is particularly suited is the ingestion and processing of log data.

Logstash can perform tasks such as transforming unstructured data into structured data, filtering out certain types of data, and adding to the data. Once Logstash is done processing data, it can forward it to a destination. For example, altered log data can be transported to Elasticsearch. If a Logstash node fails, it can guarantee at-least-once delivery through the use of its persistent queue.