Reinforcement Learning Algorithms with Python

CMA-ES

The Covariance Matrix Adaptation Evolution Strategy, or CMA-ES for short, is an evolutionary strategy algorithm. Unlike the simpler version of the evolution strategy, it samples the new candidate solution according to a multivariate normal distribution. The name CMA comes from the fact that the dependencies between the variables are kept in a covariance matrix that has been adapted to increase or decrease the search space on the next generation.

Put simply, CMA-ES shrinks the search space by incrementally decreasing the covariance matrix in a given direction when it's confident of the space around it. Instead, CMA-ES increases the covariance matrix and thus enlarges the possible search space when it's less confident.