Rule-based analysis of the log stream

Any reasonable log management system needs to be able to achieve the following:

  • Filter logs that aren't important, and therefore should not be counted or stored. These often include log entries at the INFO or DEBUG levels (yes, these exist in production systems).
  • Analyze the log entry further and extract as much meaning and new fields as possible.
  • Enhance/update the log entry prior to storage.
  • Send notifications on when certain logs are received.
  • Correlate log events to derive new meaning.
  • Deal with changes in the log's structure and formatting.

This recipe integrates the JBoss Library and Drools into a bolt to make these goals easily achievable in a declarative and clear manner. Drools is an open source implementation ...

Get Storm Real-time Processing Cookbook now with O’Reilly online learning.

O’Reilly members experience live online training, plus books, videos, and digital content from 200+ publishers.