book

Practical Simulations for Machine Learning

by Paris Buttfield-Addison, Mars Buttfield-Addison, Tim Nugent, Jon Manning

June 2022

Beginner to intermediate

331 pages

7h 15m

English

O'Reilly Media, Inc.

Read now

Unlock full access

Resources Used in This BookAudience and ApproachOrganization of This BookUsing This BookOur TasksConventions Used in This BookUsing Code ExamplesO’Reilly Online LearningHow to Contact UsAcknowledgments
A Whole New World of MLThe DomainsSimulationSynthesisThe ToolsUnityPyTorch via Unity ML-AgentsUnity ML-Agents ToolkitUnity PerceptionThe TechniquesReinforcement LearningImitation LearningHybrid LearningSummary of TechniquesProjectsSimulation ProjectsSynthesis ProjectsSummary and Next Steps
Everybody Remembers Their First SimulationOur SimulationSetting UpCreating the Unity ProjectPackages All the Way DownThe EnvironmentThe FloorThe TargetThe AgentStarting and Stopping the AgentLetting the Agent Observe the EnvironmentLetting the Agent Take Actions in the EnvironmentGiving the Agent Rewards for Its BehaviorFinishing Touches for the AgentProviding a Manual Control System for the AgentTraining with the SimulationMonitoring the Training with TensorBoardWhen the Training Is CompleteWhat’s It All Mean?Coming Up Next
Unity PerceptionThe ProcessUsing Unity PerceptionCreating the Unity ProjectCreating a SceneGetting the Dice ModelsA Very Simple ScenePreparing for SynthesisTesting the ScenarioSetting Up Our LabelsChecking the LabelsWhat’s Next?
Setting Up the Block PusherCreating the Unity ProjectThe EnvironmentThe FloorThe WallsThe BlockThe GoalThe AgentThe EnvironmentTraining and Testing
Creating the EnvironmentThe TrackThe CarSetting Up for MLTraining the SimulationTrainingWhen the Training Is Complete
Simulation EnvironmentCreating the GroundCreating the GoalThe Name’s Ball, Agent BallThe CameraBuilding the SimulationAgent ComponentsAdding Heuristic ControlsObservations and GoalsGenerating Data and TrainingCreating Training DataConfiguring for TrainingBegin TrainingRunning with Our Trained ModelUnderstanding and Using Imitation Learning
Meet GAILDo What I Say and DoA GAIL ScenarioModifying the Agent’s ActionsModifying the ObservationsResetting the AgentUpdating the Agent PropertiesDemonstration TimeTraining with GAILRunning It and Beyond

Curriculum Learning in MLA Curriculum Learning ScenarioBuilding in UnityCreating the GroundCreating the TargetThe AgentBuilding the SimulationMaking the Agent an AgentActionsObservationsHeuristic Controls for HumansCreating the CurriculumResetting the EnvironmentCurriculum ConfigTrainingRunning ItCurriculum Versus Other ApproachesWhat’s Next?
A Simulation for CooperationBuilding the Environment in UnityCoding the AgentsCoding the Environment ManagerCoding the BlocksFinalizing the Environment and AgentsTraining for CooperationCooperative Agents or One Big Agent
Observations and Camera SensorsBuilding a Camera-Only AgentCoding the Camera-Only AgentAdding a New Camera for the AgentSeeing What the Agent’s Camera SeesTraining the Camera-Based AgentCameras and You
Python All the Way DownExperimenting with an EnvironmentWhat Can Be Done with Python?Using Your Own EnvironmentCompletely Custom TrainingWhat’s the Point of Python?
Hyperparameters (and Just Parameters)ParametersReward ParametersHyperparametersAlgorithmsUnity Inference Engine and IntegrationsUsing the ML-Agents Gym WrapperSide Channels
Adding Random Elements to the SceneRandomizing the Floor ColorRandomizing the Camera PositionWhat’s Next?
Creating the Unity EnvironmentA Perception CameraFaking It Until You Make ItUsing Synthesized Data

Content preview from Practical Simulations for Machine Learning

Chapter 2. Creating Your First Simulation

We’re going to get started by looking at a simple simulation environment: a ball agent that can roll around a platform. As we said earlier, we know it’s a lot to handle, but we think you’ll be able to cope with the levels of excitement and come through with a better understanding of machine learning and simulation with Unity.

Everybody Remembers Their First Simulation

In this chapter we’re going to build a brand-new simulation environment using Unity, create an agent, and then train that agent to accomplish a task in the environment using reinforcement learning. It’s going to be a very simple simulation environment, but it will serve to demonstrate a number of important things:

How straightforward it is to assemble a scene in Unity by using a small collection of simple objects
How to use the Unity Package Manager to import the Unity side of the Unity ML-Agents Toolkit into Unity and set up a Unity project for machine learning
How to set up a simple agent in your simulation object with the intention of enabling it to accomplish a task
How to take manual control of your agent to test the simulation environment
How to start a training run using the command-line tool (CLI) side of the Unity ML-Agents Toolkit, and how to bring up TensorBoard to monitor the training’s progress
How to bring a trained model file back into a Unity simulation environment and run the agent using the trained model