Mastering Apache Storm

Custom scheduler

In Storm, Nimbus uses a scheduler to assign tasks to the supervisors. The default scheduler aims to allocate computing resources evenly to topologies. It works well in terms of fairness among topologies, but it is impossible for users to predict the placement of topology components in the Storm cluster, regarding which component of a topology needs to be assigned to which supervisor node.

Let's consider an example. Say that we have a topology that has one spout and two bolts, and each of the components has one executor and one task. The following diagram shows the distribution of the topology if we submit the topology to a Storm cluster. Assume that the number of workers assigned to the topology is three and the number of ...