Design – Build – Run: Applied Practices and Principles for Production-Ready Software Development

20.3. Monitoring Configuration

As you can tell from the discussions so far, nothing is really cast-in-stone, except that things will differ from project to project. Things will actually differ from environment to environment as well. A Warning event in one environment may not necessarily be a Warning event in another; it could be Information or perhaps even an Error event. There's no denying that correctly instrumenting an application has a performance overhead, not to mention development and test overheads. But this needs to be balanced against effective monitoring and support. Where possible, the application should support a flexible approach to monitoring through configuration.

20.3.1. Performance Counter Configuration

The configuration should allow performance counters to be switched on or off. The application shouldn't update counters that have been switched off in the configuration. This greatly helps with testing because the counters can be switched on and the values can be sampled to provide reports. However, the counters can be switched off in production to improve the overall performance of the application. The configuration should also allow the sampling period to be modified where appropriate. For example, if a counter is being updated every 100^th transaction, this should be a configurable value. All the performance counters in the system should be well-documented and understood so that the production configuration is optimum. The configuration may change on-the-fly, ...