The Heartbeat server (HBS) is used by the system to keep track of which servers are up and which are down. Heartbeating refers to a server transmitting regular pulses to a multicast address as a signal to the other servers that it is up.
The Transmit thread is the multicast group. Any type of server that uses an HBS sends heartbeat pulses to the multicast group. If a server wants to determine the status of other servers on the network, it polls the Multicast thread regularly. The polling frequency is configurable and set to a default value of 250 milliseconds.
The RX thread specifies which servers a particular server listens to for heartbeats. Some servers receive the TX thread without keeping track of which servers are up; other servers listen for heartbeats from specific servers.
Each server has a data container that it uses to match incoming heartbeats. The matching criteria include server type, IP address, and port. Every server receives every multicast heartbeat but throws away those that do not match the list in its data container. If heartbeats are not received for servers listed in the data container, it is assumed that those servers are down.
Every 250 milliseconds, a heartbeat is received and marked. Housekeeping checks if any anticipated heartbeats fail to appear. The Housekeeping thread flags the counter for missed heartbeats and increments the counter by 1. If the counter matches the maximum missed heartbeat ...