book

Practical Simulations for Machine Learning

by Paris Buttfield-Addison, Mars Buttfield-Addison, Tim Nugent, Jon Manning

June 2022

Beginner to intermediate

331 pages

7h 15m

English

O'Reilly Media, Inc.

Read now

Unlock full access

Resources Used in This BookAudience and ApproachOrganization of This BookUsing This BookOur TasksConventions Used in This BookUsing Code ExamplesO’Reilly Online LearningHow to Contact UsAcknowledgments
A Whole New World of MLThe DomainsSimulationSynthesisThe ToolsUnityPyTorch via Unity ML-AgentsUnity ML-Agents ToolkitUnity PerceptionThe TechniquesReinforcement LearningImitation LearningHybrid LearningSummary of TechniquesProjectsSimulation ProjectsSynthesis ProjectsSummary and Next Steps
Everybody Remembers Their First SimulationOur SimulationSetting UpCreating the Unity ProjectPackages All the Way DownThe EnvironmentThe FloorThe TargetThe AgentStarting and Stopping the AgentLetting the Agent Observe the EnvironmentLetting the Agent Take Actions in the EnvironmentGiving the Agent Rewards for Its BehaviorFinishing Touches for the AgentProviding a Manual Control System for the AgentTraining with the SimulationMonitoring the Training with TensorBoardWhen the Training Is CompleteWhat’s It All Mean?Coming Up Next
Unity PerceptionThe ProcessUsing Unity PerceptionCreating the Unity ProjectCreating a SceneGetting the Dice ModelsA Very Simple ScenePreparing for SynthesisTesting the ScenarioSetting Up Our LabelsChecking the LabelsWhat’s Next?
Setting Up the Block PusherCreating the Unity ProjectThe EnvironmentThe FloorThe WallsThe BlockThe GoalThe AgentThe EnvironmentTraining and Testing
Creating the EnvironmentThe TrackThe CarSetting Up for MLTraining the SimulationTrainingWhen the Training Is Complete
Simulation EnvironmentCreating the GroundCreating the GoalThe Name’s Ball, Agent BallThe CameraBuilding the SimulationAgent ComponentsAdding Heuristic ControlsObservations and GoalsGenerating Data and TrainingCreating Training DataConfiguring for TrainingBegin TrainingRunning with Our Trained ModelUnderstanding and Using Imitation Learning
Meet GAILDo What I Say and DoA GAIL ScenarioModifying the Agent’s ActionsModifying the ObservationsResetting the AgentUpdating the Agent PropertiesDemonstration TimeTraining with GAILRunning It and Beyond

Curriculum Learning in MLA Curriculum Learning ScenarioBuilding in UnityCreating the GroundCreating the TargetThe AgentBuilding the SimulationMaking the Agent an AgentActionsObservationsHeuristic Controls for HumansCreating the CurriculumResetting the EnvironmentCurriculum ConfigTrainingRunning ItCurriculum Versus Other ApproachesWhat’s Next?
A Simulation for CooperationBuilding the Environment in UnityCoding the AgentsCoding the Environment ManagerCoding the BlocksFinalizing the Environment and AgentsTraining for CooperationCooperative Agents or One Big Agent
Observations and Camera SensorsBuilding a Camera-Only AgentCoding the Camera-Only AgentAdding a New Camera for the AgentSeeing What the Agent’s Camera SeesTraining the Camera-Based AgentCameras and You
Python All the Way DownExperimenting with an EnvironmentWhat Can Be Done with Python?Using Your Own EnvironmentCompletely Custom TrainingWhat’s the Point of Python?
Hyperparameters (and Just Parameters)ParametersReward ParametersHyperparametersAlgorithmsUnity Inference Engine and IntegrationsUsing the ML-Agents Gym WrapperSide Channels
Adding Random Elements to the SceneRandomizing the Floor ColorRandomizing the Camera PositionWhat’s Next?
Creating the Unity EnvironmentA Perception CameraFaking It Until You Make ItUsing Synthesized Data

Content preview from Practical Simulations for Machine Learning

Chapter 13. Creating More Advanced Synthesized Data

In this chapter, we’ll return to synthesis and build upon the introduction to synthesizing data using Unity’s Perception that we worked through back in Chapter 3.

Specifically, we’ll use randomizers to add a random element to the images generated from our dice, and learn how to explore the data we’re synthesizing, making use of the labels we added earlier.

Adding Random Elements to the Scene

To generate useful synthetic data, we need to add random elements to the scene. The random elements we’re going to add are:

A random floor color
A random camera position

By randomly changing the color of the floor and the position of the camera, we’ll be able to generate a variety of random images of dice, which can then be used to train an image recognition system outside of Unity to recognize dice in a huge range of situations.

We’re going to be working with the same project we ended up with way back at the end of Chapter 3, so either duplicate it or re-create it from scratch before continuing. We duplicated it and renamed it “SimpleDiceWithRandomizers.”

Tip

Don’t forget that it needs to be a 3D URP project, which is different from the projects you’ve been making throughout Part II for simulations. Refer back to “Creating the Unity Project” if you need a reminder.