Practical Simulations for Machine Learning

Chapter 11. Working with Python

In this chapter, we’re going to explore the possibilities for using Unity’s ML-Agents together with Python, in a more active way. Everything we’ve done so far has focused on the combination of Unity and (by way of ML-Agents) Python, but we’ve taken a Unity-centric approach.

This chapter is going to present a few ways you can do things from a Python-centric approach. Specifically, we’ll look at how you can use Python to interact with Unity and ML-Agents.

Python All the Way Down

We’ve been using Python throughout the book, but we’re also using a script that runs the training and connects it to Unity. PyTorch is also being used deep under the hood to do the training. Beyond that, though, it’s not really relevant that we’re using Python. It just happens to be Python that powers the scripts we’re running. It could be anything.

In this chapter, we’ll experiment more with Python and get a handle on the capabilities generated by combining Python with ML-Agents, beyond the provided scripts. We’ll prove that it’s actually Python you’re using when you run mlagents-learn to train an agent, and look beyond the provided scripts a little bit.

The environment we’ll spend most of our time poking at in this chapter, via Python, is called the GridWorld environment. It’s a simple 3x3 grid, where the agent is a blue square that needs to touch a green +, and not touch a red x. It uses an exclusively visual observation system: a camera pointed down at the grid.

Its actions ...