Deploying Node.js

Chapter 5. Monitoring Applications

Distributed systems fail often. Worse, they often fail partially. When failures occur during operations responsible for altering a system's state (for instance, a write or delete operation), how can the correct state be recovered, especially when these operations are concurrent? To make matters even worse, some operations fail silently. Partial failures, then, can put applications in an indeterminate state. It is difficult to predict how an opaque system will behave.

Consider this quote from The Datacenter as a Computer: An Introduction to the Design of Warehouse-Scale Machines:

"Suppose a cluster has ultra-reliable server nodes with a stellar mean time between failures (MTBF) of 30 years (10,000 days)—well beyond ...