book

Practical Simulations for Machine Learning

by Paris Buttfield-Addison, Mars Buttfield-Addison, Tim Nugent, Jon Manning

June 2022

Beginner to intermediate

331 pages

7h 15m

English

O'Reilly Media, Inc.

Read now

Unlock full access

Resources Used in This BookAudience and ApproachOrganization of This BookUsing This BookOur TasksConventions Used in This BookUsing Code ExamplesO’Reilly Online LearningHow to Contact UsAcknowledgments
A Whole New World of MLThe DomainsSimulationSynthesisThe ToolsUnityPyTorch via Unity ML-AgentsUnity ML-Agents ToolkitUnity PerceptionThe TechniquesReinforcement LearningImitation LearningHybrid LearningSummary of TechniquesProjectsSimulation ProjectsSynthesis ProjectsSummary and Next Steps
Everybody Remembers Their First SimulationOur SimulationSetting UpCreating the Unity ProjectPackages All the Way DownThe EnvironmentThe FloorThe TargetThe AgentStarting and Stopping the AgentLetting the Agent Observe the EnvironmentLetting the Agent Take Actions in the EnvironmentGiving the Agent Rewards for Its BehaviorFinishing Touches for the AgentProviding a Manual Control System for the AgentTraining with the SimulationMonitoring the Training with TensorBoardWhen the Training Is CompleteWhat’s It All Mean?Coming Up Next
Unity PerceptionThe ProcessUsing Unity PerceptionCreating the Unity ProjectCreating a SceneGetting the Dice ModelsA Very Simple ScenePreparing for SynthesisTesting the ScenarioSetting Up Our LabelsChecking the LabelsWhat’s Next?
Setting Up the Block PusherCreating the Unity ProjectThe EnvironmentThe FloorThe WallsThe BlockThe GoalThe AgentThe EnvironmentTraining and Testing
Creating the EnvironmentThe TrackThe CarSetting Up for MLTraining the SimulationTrainingWhen the Training Is Complete
Simulation EnvironmentCreating the GroundCreating the GoalThe Name’s Ball, Agent BallThe CameraBuilding the SimulationAgent ComponentsAdding Heuristic ControlsObservations and GoalsGenerating Data and TrainingCreating Training DataConfiguring for TrainingBegin TrainingRunning with Our Trained ModelUnderstanding and Using Imitation Learning
Meet GAILDo What I Say and DoA GAIL ScenarioModifying the Agent’s ActionsModifying the ObservationsResetting the AgentUpdating the Agent PropertiesDemonstration TimeTraining with GAILRunning It and Beyond

Curriculum Learning in MLA Curriculum Learning ScenarioBuilding in UnityCreating the GroundCreating the TargetThe AgentBuilding the SimulationMaking the Agent an AgentActionsObservationsHeuristic Controls for HumansCreating the CurriculumResetting the EnvironmentCurriculum ConfigTrainingRunning ItCurriculum Versus Other ApproachesWhat’s Next?
A Simulation for CooperationBuilding the Environment in UnityCoding the AgentsCoding the Environment ManagerCoding the BlocksFinalizing the Environment and AgentsTraining for CooperationCooperative Agents or One Big Agent
Observations and Camera SensorsBuilding a Camera-Only AgentCoding the Camera-Only AgentAdding a New Camera for the AgentSeeing What the Agent’s Camera SeesTraining the Camera-Based AgentCameras and You
Python All the Way DownExperimenting with an EnvironmentWhat Can Be Done with Python?Using Your Own EnvironmentCompletely Custom TrainingWhat’s the Point of Python?
Hyperparameters (and Just Parameters)ParametersReward ParametersHyperparametersAlgorithmsUnity Inference Engine and IntegrationsUsing the ML-Agents Gym WrapperSide Channels
Adding Random Elements to the SceneRandomizing the Floor ColorRandomizing the Camera PositionWhat’s Next?
Creating the Unity EnvironmentA Perception CameraFaking It Until You Make ItUsing Synthesized Data

Content preview from Practical Simulations for Machine Learning

Chapter 1. Introducing Synthesis and Simulation

The world is hungry for data. Machine learning and artificial intelligence are some of the most data-hungry domains around. Algorithms and models are growing ever bigger, and the real world is insufficient. Manual creation of data and real-world systems are not scalable, and we need new approaches. That’s where Unity, and software traditionally used for video game development, steps in.

This book is all about synthesis and simulation, and leveraging the power of modern video game engines for machine learning. Combining machine learning with simulations and synthetic data sounds relatively straightforward on the surface, but the reality is the idea of including video game technology in the serious business world of machine learning scares an unreasonable number of companies and businesses away from the idea.

We hope this book will steer you into this world and alleviate your concerns. Three of the authors of this book are video game developers with a significant background in computer science, and one is a serious machine learning and data scientist. Our combined perspectives and knowledge, built over many years in a variety of industries and approaches, are presented here for you.

This book will take you on a journey through the approaches and techniques that can be used to build and train machine learning systems using, and using data generated by, the Unity video game engine. There are two distinct domains in this book: simulation ...