O'Reilly logo

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Microsoft HoloLens By Example

Book Description

Get to grips with HoloLens development as you create mixed reality apps from scratch

About This Book

  • Create awesome Augmented Reality (AR) apps for the Microsoft HoloLens platform
  • Unleash the power of Unity SDK for HoloLens to create next generation AR apps
  • Explore the exciting world of gesture control, visual mapping, voice command for apps, and many more cutting edge possibilities with HoloLens

Who This Book Is For

This book is for developers who have some experience with programming in any of the major languages such as C#, C++, and so on. You do need any knowledge of Augmented Reality development.

What You Will Learn

  • Extend the computing experience beyond the flat glass screen by placing and embedding virtual objects holograms) into the physical world
  • Interact with the holograms using gaze, gestures, and voice
  • Enhance the experience with spatial sound
  • Allow multiple users to naturally collaborate with each other
  • Integrate voice commands into your own HoloLens projects
  • Experiment with techniques to better understand the real world
  • Implement a user interface in Mixed Reality
  • Blend the virtual and physical world by making the holograms interact and react to the physical environment

In Detail

Are you a developer who is fascinated with Microsoft HoloLens and its capabilities? Do you want to learn the intricacies of working with the HoloLens SDK and create your own apps? If so, this is the book for you.

This book introduces and demystifies the HoloLens platform and introduces new ways you can interact with computers (Mixed Reality). It will teach you the important concepts, get you excited about the possibilities, and give you the tools to continue exploring and experimenting. You will go through the journey of creating four independent examples throughout the book, two using DirectX and two using Unity. You will learn to implement spatial mapping and gesture control, incorporate spatial sound, and work with different types of input and gaze. You will also learn to use the Unity 5 SDK for HoloLens and create apps with it.

Collectively, the apps explore the major concepts of HoloLens, but each app is independent, giving you the flexibility to choose where to start (and end).

Style and approach

This book takes an example-based approach where you’ll build AR apps with increasing difficulty. You will learn more about HoloLens platform as well as AR app development in general.

Downloading the example code for this book. You can download the example code files for all Packt books you have purchased from your account at http://www.PacktPub.com. If you purchased this book elsewhere, you can visit http://www.PacktPub.com/support and register to have the code file.

Table of Contents

  1. Preface
    1. What this book covers
    2. What you need for this book
    3. Who this book is for
    4. Conventions
    5. Reader feedback
    6. Customer support
      1. Downloading the example code
      2. Downloading the color images of this book
      3. Errata
      4. Piracy
      5. Questions
  2. Enhancing Reality
    1. A brief look at the past
    2. Defining mixed reality
      1. Designing for mixed reality
        1. Identifying the type of experience you want to create
      2. Understanding the environment
        1. Thinking in terms of the real world
      3. Interacting in mixed reality
    3. Summary
  3. Tagging the World Using DirectX
    1. Creating a mixed reality app
    2. Project setup
      1. Setting up Azure
      2. Creating a group 
    3. The project
      1. Adding the Face API key and App Capabilities
      2. Grabbing frames
      3. Capturing frames
      4. Rendering holographic labels
      5. Positioning our label
    4. Summary
  4. Assistant Item Finder Using DirectX
    1. Project setup
      1. Adding the Computer Vision API service
    2. The project
      1. Leaving breadcrumbs
        1. Node
        2. Edges 
        3. Entity
        4. Leaving a trail
        5. Seeing the environment 
          1. FrameGrabber
          2. Capturing frames 
          3. FrameAnalyzer
          4. Integrating the FrameAnalyzer 
          5. Constructing a path 
          6. Getting assistance 
    3. Summary
  5. Building Paper Trash Ball in Unity
    1. Configuring a Unity project for HoloLens
    2. Creating our first Mixed Reality scene
    3. Mapping our environment
      1. Spatial mapping API
      2. Spatial mapping components
      3. Placing our bin
        1. Managing state
        2. Implementing gaze
        3. Finding a "good" place for the bin
        4. Placing the bin
      4. Nothing but net - throwing paper
    4. Summary
  6. Building Paper Trash Ball Using Holotoolkit in Unity
    1. Adding HoloToolkit to your project
    2. Keeping the user informed
    3. Mapping our environment
    4. Procedurally placing our bin
      1. SurfaceMeshesToPlanes
        1. Placing holograms
      2. Spatial understanding
      3. User placement versus procedural placement
    5. Nothing but net - throwing paper
    6. Summary
  7. Interacting with Holograms Using Unity
    1. Project setup
      1. Interacting with gestures
      2. Voice
        1. Keyword recognizer
        2. Grammar Recognizer using Speech Recognition Grammar Specification (SRGS)
    2. Summary
  8. Collaboration with HoloLens Using Unity
    1. Project setup 
      1. Installing and using the Blender plugin
      2. Unity starter project 
    2. The project
      1. Connecting to BlenderLIVE
      2. Understanding and using the real world
      3. Shared space
      4. Assisted discovery with spatial sound 
    3. Summary
  9. Developing a Multiplayer Game Using Unity
    1. Multiplayer networking
    2. The project
      1. Project setup
      2. Discovery
      3. Communication
      4. Shared space
      5. Coordinating
    3. Summary
  10. Deploying Your Apps to the HoloLens Device and Emulator
    1. Rapid feedback with Unity's Tooling
      1. Holographic Simulation
      2. Holographic Remoting  
      3. Exporting from Unity to Visual Studio 
    2. HoloLens emulator
      1. Deploying to the emulator
      2. Deploying to the device
    3. Windows Device Portal 
    4. Summary