Despite popular forays into augmented and virtual reality in recent years, spatial computing still sits on the cusp of mainstream use. Developers, artists, and designers looking to enter this field today have few places to turn for expert guidance. In this book, Erin Pangilinan, Steve Lukas, and Vasanth Mohan examine the AR and VR development pipeline and provide hands-on practice to help you hone your skills.
Through step-by-step tutorials, you’ll learn how to build practical applications and experiences grounded in theory and backed by industry use cases. In each section of the book, industry specialists, including Timoni West, Victor Prisacariu, and Nicolas Meuleau, join the authors to explain the technology behind spatial computing.
In three parts, this book covers:
- Art and design: Explore spatial computing and design interactions, human-centered interaction and sensory design, and content creation tools for digital art
- Technical development: Examine differences between ARKit, ARCore, and spatial mapping-based systems; learn approaches to cross-platform development on head-mounted displays
- Use cases: Learn how data and machine learning visualization and AI work in spatial computing, training, sports, health, and other enterprise applications
Table of Contents
- I. Design and Art Across Digital Realities
1. How Humans Interact with Computers
- Common Term Definition
- Modalities Through the Ages: Pre-Twentieth Century
- Modalities Through the Ages: Through World War II
- Modalities Through the Ages: Post-World War II
- Modalities Through the Ages: The Rise of Personal Computing
- Modalities Through the Ages: Computer Miniaturization
- Why Did We Just Go Over All of This?
- New Modalities
- The Current State of Modalities for Spatial Computing Devices
- Current Controllers for Immersive Computing Systems
- A Note on Hand Tracking and Hand Pose Recognition
2. Designing for Our Senses, Not Our Devices
- Envisioning a Future
- Sensory Technology Explained
- So, Who Are We Building This Future For?
- Sensory Design
- Five Sensory Principles
- Adobe’s AR Story
- II. How eXtended Reality Is Changing Digital Art
- 3. Virtual Reality for Art
- 4. 3D Art Optimization
- III. Hardware, SLAM, Tracking
5. How the Computer Vision That Makes Augmented Reality Possible Works
- Who Are We?
- A Brief History of AR
- How and Why to Select an AR Platform
- How Does Multiplayer AR Work?
- What’s the Difficult Part?
- How Does Relocalization Work?
- What’s the State of the Art in Research (and Coming Soon to Consumer)?
- Can the Relocalization Problem Really Be Solved for Consumers?
- Can’t Google or Apple Just Do This?
- Relocalization != Multiplayer; It’s Also Critical for…
- How Is Relocalization Really Being Done Today in Apps?
- Other Development Considerations
The AR Cloud
- What I Envision When I Think About the AR Cloud
- How Big a Deal Is the AR Cloud?
- So, Just What Is This AR Cloud?
- Why All the “Meh” from the Press for ARKit and ARCore?
- What’s Missing to Make a Great AR App?
- Is Today’s Mobile Cloud up to the Job?
- Is ARKit (or ARCore) Useless Without the AR Cloud?
- The Dawn of the AR Cloud
- The Bigger Picture—Privacy and AR Cloud Data
- IV. Creating Cross-Platform Augmented Reality and Virtual Reality
- 6. Virtual Reality and Augmented Reality: Cross-Platform Theory
- 7. Virtual Reality Toolkit: Open Source Framework for the Community
- 8. Three Virtual Reality and Augmented Reality Development Best Practices
- V. Enhancing Data Representation: Data Visualization and Artificial Intelligence in Spatial Computing
9. Data and Machine Learning Visualization Design and Development in Spatial Computing
- Understanding Data Visualization
- Principles for Data and Machine Learning Visualization in Spatial Computing
- Why Data and Machine Learning Visualization Works in Spatial Computing
- 2D Data Visualizations versus 3D Data Visualization in Spatial Computing
- Interactivity in Data Visualizations in Spatial Computing
- Failures in Data Visualization Design
- Good Data Visualization Design Optimize 3D Spaces
- “Savings in Time Feel Like Simplicity”
- Data Representations, Infographics, and Interactions
- Defining Distinctions in Data Visualization and Big Data or Machine Learning Visualizations
- How to Create Data Visualization: Data Visualization Creation Pipeline
- WebXR: Building Data Visualizations for the Web
- Data Visualization Challenges in XR
- Data Visualization Industry Use Case Examples of Data Visualizations
- 3D Reconstruction and Direct Manipulation of Real-World Data: Anatomical Structures in XR
- Data Visualization Is for Everyone: Open Source–Based Data Visualization in XR
- Hands-On Tutorials: How to Create Data Visualization in Spatial Computing
- How to Create Data Visualization: Resources
10. Character AI and Behaviors
- Current Practice: Reactive AI
- More Intelligence in the System: Deliberative AI
- Machine Learning
- VI. Use Cases in Embodied Reality
11. The Virtual and Augmented Reality Health Technology Ecosystem
- VR/AR Health Technology Application Design
- Standard UX Isn’t Intuitive
- Tutorial: Insight Parkinson’s Experiment
- Case Studies from Leading Academic Institutions
- 12. The Fan Experience: SportsXR
13. Virtual Reality Enterprise Training Use Cases
- Introduction: The Importance of Enterprise Training
- Does VR Training Work?
- Use Case: Flood House Training
- What Is VR Training Good for? R.I.D.E.
- What Makes Good VR Training?
- Spherical Video
- Use Case: Factory Floor Training
- The Role of Narrative
- Use Case: Store Robbery Training
- The Future of XR Training: Beyond Spherical Video
- Use Case: Soft Skills Training
- The Future: Photogrammetry
- The Future: Light Fields
- The Future: AR Training
- The Future: Voice Recognition
- The Future: The Ideal Training Scenario
- Title: Creating Augmented and Virtual Realities
- Release date: April 2019
- Publisher(s): O'Reilly Media, Inc.
- ISBN: 9781492044192