Monitoring performance metrics
Now that we have covered a few options for process monitoring, how do you know if your application is actually doing the work you think it is? On many occasions I've seen a stuck
syslog-ng process that appeared to be running, but it just wasn't sending any data. I'm not picking on
syslog-ng specifically; all software does this when conditions occur that it isn't designed to deal with.
When talking about Flume data flows, you need to monitor the following:
- Data entering sources is within expected rates
- Data isn't overflowing your channels
- Data is exiting sinks at an expected rates
Flume has a pluggable monitoring framework, but as mentioned at the beginning of the chapter, it is still very much a work in progress. That ...