The efficiency and stability of TRPO allowed us to test it on new and more complex environments. We applied it on Roboschool. Roboschool and its Mujoco counterpart are often used as a testbed for algorithms that are able to control complex agents with continuous actions, such as TRPO. Specifically, we tested TRPO on RoboschoolWalker2d, where the task of the agent is to learn to walk as fast as possible. This environment is shown in the following figure. The environment terminates whenever the agent falls or when more than 1,000 timesteps have passed since the start. The state is encoded in a Box class of size 22 and the agent is controlled with 6 float values with a range of :
Figure 7.6. Render of the RoboschoolWalker2d ...
