The standard environment for building continuous control agents is the Mujoco environment. Mujoco stands for Multi-Joint dynamics with Contract and it is a full physics environment for training robotic or simulation agents. This environment provides a number of simulations that challenge some form of robotic control agent to perform a task, such as walking, crawling, and implementing several other physics control-based tasks. An example of the diversity of these environments is summarized well in the following image, which has been extracted from the Mujoco home page:
Extract of example environments ...
