PyTorch 1.x Reinforcement Learning Cookbook

How to do it...

We perform SARSA to solve the Windy Gridworld environment as follows:

1. Import PyTorch and WindyGridworldEnvmodule (assuming that it is in a file called windy_gridworld.py), and create an instance of the Windy Gridworld environment:

>>> import torch>>> from windy_gridworld import WindyGridworldEnv>>> env = WindyGridworldEnv()

2. Let's start by defining the epsilon-greedy behavior policy:

>>> def gen_epsilon_greedy_policy(n_action, epsilon): ...     def policy_function(state, Q): ...         probs = torch.ones(n_action) * epsilon / n_action ...         best_action = torch.argmax(Q[state]).item() ...         probs[best_action] += 1.0 - epsilon ...         action = torch.multinomial(probs, 1).item() ...         return action ...     return policy_function

3. We specify the ...