book

Creating Augmented and Virtual Realities

by Erin Pangilinan, Steve Lukas, Vasanth Mohan

April 2019

Intermediate to advanced

370 pages

10h 23m

English

O'Reilly Media, Inc.

Read now

Unlock full access

A Note on TerminologyWhy We Wrote This BookWhat We Didn’t Cover in This BookHow This Book Is OrganizedConventions Used in This BookUsing Code ExamplesO’Reilly Online LearningHow to Contact UsAcknowledgments
Common Term DefinitionIntroductionModalities Through the Ages: Pre-Twentieth CenturyModalities Through the Ages: Through World War IIModalities Through the Ages: Post-World War IIModalities Through the Ages: The Rise of Personal ComputingModalities Through the Ages: Computer MiniaturizationWhy Did We Just Go Over All of This?Types of Common HCI ModalitiesNew ModalitiesThe Current State of Modalities for Spatial Computing DevicesCurrent Controllers for Immersive Computing SystemsBody Tracking TechnologiesA Note on Hand Tracking and Hand Pose RecognitionVoice, Hands, and Hardware Inputs over the Next Generation
Envisioning a FutureSensory Technology ExplainedSo, Who Are We Building This Future For?The Role of Women in AISensory DesignAn IntroductionFive Sensory Principles1. Intuitive Experiences Are Multisensory2. 3D Will Be Normcore3. Designs Become Physical Nature4. Design for the Uncontrollable5. Unlock the Power of Spatial CollaborationAdobe’s AR StoryConclusion
A More Natural Way of Making 3D ArtVR for Animation
IntroductionOptions to ConsiderIdeal SolutionTopologyBakingDraw CallsUsing VR Tools for Creating 3D ArtAcquiring 3D Models Versus Making Them from ScratchSummary
Who Are We?A Brief History of ARHow and Why to Select an AR PlatformI’m a Developer, What Platform Should I Use and Why?Performance Is StatisticsIntegrating Hardware and SoftwareOptical CalibrationInertial CalibrationThe Future of TrackingThe Future of AR Computer VisionMappingHow Does Multiplayer AR Work?What’s the Difficult Part?How Does Relocalization Work?What’s the State of the Art in Research (and Coming Soon to Consumer)?Can the Relocalization Problem Really Be Solved for Consumers?Can’t Google or Apple Just Do This?Relocalization != Multiplayer; It’s Also Critical for…How Is Relocalization Really Being Done Today in Apps?PlatformsApple’s ARKitSome Mysteries ExplainedIsn’t ARCore Just Tango-Lite?So, Should I Build on ARCore Now?What About Tango, Hololens, Vuforia, and Others?Other Development ConsiderationsLightingMultiplayer AR—Why It’s Quite DifficultHow Do People Connect Through AR?How Do AR Apps Connect to the World and Know Where They Really Are?How Do AR Apps Understand and Connect to Things in the Real World?The AR CloudWhat I Envision When I Think About the AR CloudHow Big a Deal Is the AR Cloud?So, Just What Is This AR Cloud?Why All the “Meh” from the Press for ARKit and ARCore?What’s Missing to Make a Great AR App?Is Today’s Mobile Cloud up to the Job?Is ARKit (or ARCore) Useless Without the AR Cloud?The Dawn of the AR CloudThe Bigger Picture—Privacy and AR Cloud DataGlossary

Why Cross-Platform?The Role of Game EnginesUnderstanding 3D GraphicsThe Virtual CameraDegrees of FreedomPortability Lessons from Video Game DesignSimplifying the Controller InputDevelopment Step 1: Designing the Base InterfaceDevelopment Step 2: Platform IntegrationSummary
What Is VRTK and Why People Use It?The History of VRTKWelcome to the SteamVR Unity ToolkitVRTK v4The Future of VRTKThe Success of VRTKGetting Started with VRTK 4
Developing for Virtual Reality and Augmented Reality Is DifficultHandling LocomotionLocomotion in VRLocomotion in AREffective Use of AudioAudio in VRAudio in ARCommon Interactions ParadigmsInventory for VRAugmented Reality RaycastsConclusion
IntroductionUnderstanding Data VisualizationPrinciples for Data and Machine Learning Visualization in Spatial ComputingWhy Data and Machine Learning Visualization Works in Spatial ComputingThe Evolution of Data Visualization Design with the Emergence of XR2D and 3D Data Represented in XR2D Data Visualizations versus 3D Data Visualization in Spatial ComputingInteractivity in Data Visualizations in Spatial ComputingAnimationFailures in Data Visualization DesignGood Data Visualization Design Optimize 3D Spaces“Savings in Time Feel Like Simplicity”Data Representations, Infographics, and InteractionsWhat Qualifies as Data Visualization?Defining Distinctions in Data Visualization and Big Data or Machine Learning VisualizationsHow to Create Data Visualization: Data Visualization Creation PipelineWebXR: Building Data Visualizations for the WebData Visualization Challenges in XRData Visualization Industry Use Case Examples of Data Visualizations3D Reconstruction and Direct Manipulation of Real-World Data: Anatomical Structures in XRA Closer Look at Glass BrainTVA Surg Medical Imaging VR ModuleData Visualization Is for Everyone: Open Source–Based Data Visualization in XRProtein Data VisualizationHands-On Tutorials: How to Create Data Visualization in Spatial ComputingHow to Create Data Visualization: ResourcesConclusionReferencesResourcesData Visualization ToolsMachine Learning Visualization ToolsData Journalism VisualizationsGlossary
IntroductionBehaviorsCurrent Practice: Reactive AIAdaptabilityComplexity and UniversalityFeasibilityMore Intelligence in the System: Deliberative AIMachine LearningReinforcement LearningDeep Reinforcement LearningImitation LearningCombining Automated Planning and Machine LearningApplicationsConclusionReferences
VR/AR Health Technology Application DesignStandard UX Isn’t IntuitivePick a Calm EnvironmentConvenienceTutorial: Insight Parkinson’s ExperimentWhat Insight DoesHow It Was BuiltCompaniesPlanning and GuidanceExperiences Designed for Medical EducationExperiences Designed for Use by PatientsProactive HealthCase Studies from Leading Academic Institutions
IntroductionPart 1: Five Key Principles of AR and VR for SportsNothing Is LivePart 2: The Next Evolution of Sports ExperiencesPart 3: Making the FutureOwnershipFinal ThoughtConclusion
Introduction: The Importance of Enterprise TrainingDoes VR Training Work?Use Case: Flood House TrainingWhat Is VR Training Good for? R.I.D.E.What Makes Good VR Training?Spherical VideoThe Benefits of Spherical VideoThe Challenges of Spherical VideoInteractions with Spherical VideoUse Case: Factory Floor TrainingThe Role of NarrativeUse Case: Store Robbery TrainingThe Future of XR Training: Beyond Spherical VideoComputer GraphicsUse Case: Soft Skills TrainingThe Future: PhotogrammetryThe Future: Light FieldsThe Future: AR TrainingThe Future: Voice RecognitionThe Future: The Ideal Training ScenarioReferences

Content preview from Creating Augmented and Virtual Realities

Chapter 12. The Fan Experience: SportsXR

Marc Rowley

Introduction

This is truly an amazing time to be a sports fan, and thanks to technology, the future of sports is unlimited. This chapter focuses on augmented reality (AR), virtual reality (VR), and sports. The connections that we as fans have with sports have driven developments in media and technology over the past few years at a ferocious pace. Sports has been one of the most consumed content categories in the global digital media marketplace and is moving technology forward for more sports AR and VR experiences.

Here are the ground rules that developers need to know:

Sports are events in which rules are set, contestants compete, and there is a result.
AR and VR use technology to create and enhance content. The best example of this is the “First-and-Ten line” from Sportvision in 1998.
Live action matters, it creates a sense of wonder, an anticipation, a wanting to not miss out.

To explore this juggernaut more thoroughly, this chapter is separated into three parts:

Five key principles of AR and VR for sports
The next evolution of sports experiences
Making the future

First, proper introductions. I’m Marc Rowley, and I consider myself an AR/VR pioneer, having worked on live AR in sports for more than 20 years. I have five Emmy awards, multiple global innovation awards, and I have founded several AR companies. As I see it, the best moment for a storyteller is to see the audience’s reaction when you show them something ...