After expansion, the rest of the game is played by randomly choosing subsequent moves. This is also commonly referred to as the playout or rollout. Depending on the game, some heuristics may be applied to choose the next move. For example, in DeepBlue, simulations rely on handcrafted heuristics to select the next move intelligently rather than randomly. This is also called heavy rollouts. While such rollouts provide more realistic games, they are often computationally expensive, which can slow down the learning of the MCTS tree:
Figure 3: Simulation
In our preceding toy example, we expand a node and play until the very end of the ...
