It is natural to feel that more, faster CPU cores are better, but with Ceph this is usually the least effective place to spend extra money. The first and largest bottlenecks a cluster faces under load are usually saturating the storage drives or the replication network. Ceph daemons are multi threaded, so solving for core count is usually better than throwing money up the price/GHz curve. Rules of thumb can be tricky and opinions here vary, especially as CPUs evolve, but as a minimum this author suggests the following:
- MON nodes: four cores
- RGW nodes: eight cores
- MDS nodes: four-eight cores at a higher clock rate
- OSD nodes: one core-GHz per HDD, two per SSD, five per NVMe. Round up if using dmcrypt or compression.
OSD systems ...