Chapter 4. Planning historical data collection in large scale environments 81
attribute groups and the summarized data for these attribute groups should go
into the calculation of the database size.
The length of time that the data will be kept in the database should also be taken
into account. Note that often the most amount of data comes from the detailed
data, but after summarized data has been created from the detailed data, usually
summarized data (not the detailed data) is used in the reports. However, the
detailed data is held in the database for a long time. Therefore, you need to
make sure that you set the proper summarization and pruning settings in the
historical data collection dialog, as shown in Figure 4-1 on page 63.
The data collection interval for an attribute group can be specified as either 5, 15,
30 or 60 minutes. (In IBM Tivoli Monitoring V6.2, a 1-minute data collection will
In configuring historical data collection for an agent attribute group, specify the
longest data collection interval that will still provide useful data. More frequent
data collection results in more rows to be inserted into the warehouse, more
database storage usage, and more work for the Summarization and Pruning
agent. Using a 5-minute data collection interval generates three times as many
rows as a 15-minute interval, and 12 times as many rows as an hourly data
4.3.1 Use the Warehouse Load Projections spreadsheet to estimate
and control the Tivoli Warehouse database size
IBM Tivoli Monitoring Installation and Setup Guide, GC32-9407, contains
information about how to estimate the disk space requirements for the Tivoli Data
Warehouse. As described, the estimation process is executed in these steps:
1. Determine the number of detailed records per day for the attribute group.
– (60 / collection interval) * 24 * (# instances of each interval)
2. Determine the hard disk footprint (in KB) for the attribute group.
– (# detailed records) * (attribute group detailed record size) / 1024
Important: We strongly discourage you from turning on the default historical
data collection. In fact, in IBM Tivoli Monitoring V6.2, the default settings tab in
the Summarization and Pruning agent configuration dialog box will be
The reason for this is because settings for every single attribute group should
depend on the reporting and visualization requirements. Therefore, a default
value should not be used.
82 IBM Tivoli Monitoring: Implementation and Performance Optimization for Large Scale Environments
3. Determine the amount of detailed data (in MB) for the attribute group.
– (hard disk footprint) * (# of agents) * (# days of detailed data) / 1024
4. Calculate the amount of aggregate data (in MB) for the attribute group.
– ((# hourly) + (# daily) + (# weekly) + (# monthly) + (# quarterly) + (#
yearly)) * (# instances of each interval) * (attribute group aggregate record
size) / 1048576
5. Determine the estimated size of your database.
– (detailed record size) + (aggregate record size)
The IBM Redbooks publication Tivoli Management Services Warehouse and
Reporting, SG24-7290, contains a detailed projection with the Warehouse Load
Projection spreadsheet that covers all these steps. The spreadsheet can be
downloaded from the OPAL Web site for IBM Tivoli monitoring:
At the site, search for the topic “Warehouse Load Projections spreadsheet.”
The Warehouse Load Projections spreadsheet has been created to simplify the
task of producing the disk space estimate for the Tivoli Data Warehouse. It
implements the steps described, and automates the process.
This spreadsheet includes the attribute group information for over 50 different
agent types, and allows the user to perform “what-if” exercises to see the effects
of different historical data collection configuration options. The spreadsheet
includes two predefined charts showing the contribution of each agent type to the
total Tivoli Data Warehouse disk space estimate.
Because it is implemented in a standard spreadsheet format, other charts can be
generated easily. Projections produced by this spreadsheet should be viewed as