Log stream processors
Stream processing technologies are optionally used for processing log streams on the fly. For example, if a 404 error is continuously received as a response to a particular service call, it means there is something wrong with the service. Such situations have to be handled as soon as possible. Stream processors are pretty handy in such cases, as they are capable of reacting to certain streams of events compared to traditional reactive analysis.
A typical architecture used for stream processing is a combination of Flume and Kafka together, with either Storm or Spark Streaming. Log4j has Flume appenders that are useful for collecting log messages. These messages will be pushed into distributed Kafka message queues. The ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access