Learning Apache Apex
by Ananth Gundabattula, Thomas Weise, Munagala V. Ramanath, David Yan, Kenneth Knowles
Affinity and anti-affinity
A related topic is operator affinity and anti-affinity. There are circumstances when we want a set of operators to run on the same (affinity) or different (anti-affinity) node, container, or thread. For example, suppose we have an upstream operator A and a downstream operator B that is not a direct successor of A in the DAG. If they are both heavy consumers of CPU resources, it makes sense to allocate them on different nodes (anti-affinity). Stream locality cannot achieve this effect, since there is no stream connecting them. Another situation is where the application is computing latencies between operators A and B. Simply looking at the system clock may not be adequate, since the clocks on different nodes may ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access