book

Practical Simulations for Machine Learning

by Paris Buttfield-Addison, Mars Buttfield-Addison, Tim Nugent, Jon Manning

June 2022

Beginner to intermediate

331 pages

7h 15m

English

O'Reilly Media, Inc.

Read now

Unlock full access

Resources Used in This BookAudience and ApproachOrganization of This BookUsing This BookOur TasksConventions Used in This BookUsing Code ExamplesO’Reilly Online LearningHow to Contact UsAcknowledgments
A Whole New World of MLThe DomainsSimulationSynthesisThe ToolsUnityPyTorch via Unity ML-AgentsUnity ML-Agents ToolkitUnity PerceptionThe TechniquesReinforcement LearningImitation LearningHybrid LearningSummary of TechniquesProjectsSimulation ProjectsSynthesis ProjectsSummary and Next Steps
Everybody Remembers Their First SimulationOur SimulationSetting UpCreating the Unity ProjectPackages All the Way DownThe EnvironmentThe FloorThe TargetThe AgentStarting and Stopping the AgentLetting the Agent Observe the EnvironmentLetting the Agent Take Actions in the EnvironmentGiving the Agent Rewards for Its BehaviorFinishing Touches for the AgentProviding a Manual Control System for the AgentTraining with the SimulationMonitoring the Training with TensorBoardWhen the Training Is CompleteWhat’s It All Mean?Coming Up Next
Unity PerceptionThe ProcessUsing Unity PerceptionCreating the Unity ProjectCreating a SceneGetting the Dice ModelsA Very Simple ScenePreparing for SynthesisTesting the ScenarioSetting Up Our LabelsChecking the LabelsWhat’s Next?
Setting Up the Block PusherCreating the Unity ProjectThe EnvironmentThe FloorThe WallsThe BlockThe GoalThe AgentThe EnvironmentTraining and Testing
Creating the EnvironmentThe TrackThe CarSetting Up for MLTraining the SimulationTrainingWhen the Training Is Complete
Simulation EnvironmentCreating the GroundCreating the GoalThe Name’s Ball, Agent BallThe CameraBuilding the SimulationAgent ComponentsAdding Heuristic ControlsObservations and GoalsGenerating Data and TrainingCreating Training DataConfiguring for TrainingBegin TrainingRunning with Our Trained ModelUnderstanding and Using Imitation Learning
Meet GAILDo What I Say and DoA GAIL ScenarioModifying the Agent’s ActionsModifying the ObservationsResetting the AgentUpdating the Agent PropertiesDemonstration TimeTraining with GAILRunning It and Beyond

Curriculum Learning in MLA Curriculum Learning ScenarioBuilding in UnityCreating the GroundCreating the TargetThe AgentBuilding the SimulationMaking the Agent an AgentActionsObservationsHeuristic Controls for HumansCreating the CurriculumResetting the EnvironmentCurriculum ConfigTrainingRunning ItCurriculum Versus Other ApproachesWhat’s Next?
A Simulation for CooperationBuilding the Environment in UnityCoding the AgentsCoding the Environment ManagerCoding the BlocksFinalizing the Environment and AgentsTraining for CooperationCooperative Agents or One Big Agent
Observations and Camera SensorsBuilding a Camera-Only AgentCoding the Camera-Only AgentAdding a New Camera for the AgentSeeing What the Agent’s Camera SeesTraining the Camera-Based AgentCameras and You
Python All the Way DownExperimenting with an EnvironmentWhat Can Be Done with Python?Using Your Own EnvironmentCompletely Custom TrainingWhat’s the Point of Python?
Hyperparameters (and Just Parameters)ParametersReward ParametersHyperparametersAlgorithmsUnity Inference Engine and IntegrationsUsing the ML-Agents Gym WrapperSide Channels
Adding Random Elements to the SceneRandomizing the Floor ColorRandomizing the Camera PositionWhat’s Next?
Creating the Unity EnvironmentA Perception CameraFaking It Until You Make ItUsing Synthesized Data

Content preview from Practical Simulations for Machine Learning

Preface

Welcome to Practical Simulations for Machine Learning! This book combines two of our favorite things: video game engines and artificial intelligence. We hope you enjoy reading it as much as we enjoyed writing it.

Specifically, this book explores the use of Unity, a product that used to be called a game engine but now likes to be called a platform for creating and operating interactive, real-time 3D content. That’s a lot of words, but they basically boil down to this: Unity is a platform for building things in 3D, and though it has traditionally been used for video game development, it can be used to build anything that can be represented in 3D, by using a combination of 3D graphics, physics simulations, and inputs of some kind.

By combining a platform for creating and operating interactive, real-time 3D content with machine learning tools, you can use the 3D world you create to train a machine learning model, kind of like it’s the real world. It’s not actually like the real world, but it’s fun to imagine, and there are some legitimately useful connections to the real world (such as being able to generate both data for use in real-world machine learning applications, as well as models that can be transposed to physical, real-world objects, like robots).

Tip

When we say real-world, we actually mean physical.

Combining Unity with machine learning is a great way to create both simulations and synthetic data, which are the two different topics we cover in this book.