40 Algorithms Every Programmer Should Know

Strategizing multi-resource processing

Initially, large-scale algorithms were used to run on huge machines called supercomputers. These supercomputers shared the same memory space. The resources were all local—physically placed in the same machine. It means that the communications between the various processors were very fast and they were able to share the same variable through the common memory space. As the systems evolved and the need to run large-scale algorithms grew, the supercomputers evolved into Distributed Shared Memory (DSM) where each processing node used to own a portion of the physical memory. Eventually, clusters were developed, which are loosely coupled and rely on message passing among processing nodes. For large-scale algorithms, ...