In the Chapter 8, Ceph Architecture: Under the Hood, section on OSD Variance we'll explore how a Ceph cluster's complement of OSD data stores can become non-uniformly utilized and the problem this can cause. In this section we'll explore what to do about it.
The larger a cluster's OSD count grows, the wider the variance can become, approximating a bell curve. The ceph osd df utility can be used to summarize the utilization of each OSD along with an indication of how much it varies from the overall average. Here's an example of a cluster's distribution before and after we take action. Here our least full OSD is 29% full, just 80% of the cluster's mean, and the most full is over 44% full, 124% of the cluster's mean.
# ceph status ...
