HoloLens Blueprints

HoloLens Blueprints

by Abhijit Jana, Manish Sharma, Mallikarjuna Rao
Released June 2017
Publisher(s): Packt Publishing
ISBN: 9781787281943

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

Unveil the world of mixed reality with HoloLens

About This Book

  • Bring holographic insights to existing line-of-business applications, tools, and workflows

  • Focus on developing end-to-end realistic holographic application.

  • Build interactive model scripts and test them in Unity3D and holographic emulators

    • Who This Book Is For

    This book is targeted at developers and designers working on mixed-reality developments for complex integrated scenarios using HoloLens.

    What You Will Learn

  • Interact with holograms using different interaction models

  • Develop your first holographic app

  • Integrate holographic applications with cloud systems

  • Visualize data feeds coming from the cloud through holograms

  • Manage the application distribution of enterprise-enabled HoloLens

  • Integrate HoloLens applications with services deployed on Azure

  • Identify and create 3D Assets and Scenes

  • Use HoloLens to explore the Internet of Things

    • In Detail

    Do you want to create stunning applications with HoloLens? Are you a developer who is fascinated with Microsoft HoloLens and its capabilities? If so, this is the book for you.

    This book introduces and demystifies the HoloLens platform and shows you different ways of interaction with computers (mixed-reality).

    You will start your mixed-reality journey by understanding different types of digital reality. You will learn to build your first holographic app. Also, you will understand holographic application integration possibilities within Line of Business Applications using Azure. Moving ahead, you will create Integrated Solutions using IoT with HoloLens. Gradually you'll learn how to create and deploy apps on a device. You will learn to publish application to the store; if you are an enterprise developer, you will also manage and distribute applications for enterprise-enabled or domain-joined HoloLens.

    Finally, you will develop an end-to-end realistic holographic app, ranging from scenario identification to sketching, development, deployment, and, finally, production.

    Style and approach

    The book is a project-based guide to help you to create some really astonishing mixed-reality applications. It will provide end-to-end solutions and enable you to build stunning applications for HoloLens.

    Table of contents

    1. Preface
      1. What this book covers
      2. What you need for this book
      3. Who this book is for
      4. Conventions
      5. Reader feedback
      6. Customer support
        1. Downloading the example code
        2. Downloading the color images of this book
        3. Errata
        4. Piracy
        5. Questions
    2. Digital Reality - Under the Hood
      1. Virtual Reality - what it is?
        1. Realizing Virtual Reality
        2. Virtual Reality in the field
      2. Augmented Reality - what it is?
        1. Visualization techniques for Augmented Reality
          1. Screen-based
          2. Augmented Reality head-mounted globally displays
          3. Projection-based
        2. Augmented Reality in the field
      3. Mixed Reality - what it is?
        1. Mixed Reality in the field
      4. Digital Reality and the current market place
      5. Summary
    3. HoloLens – The Most Natural Way to Interact
      1. A closer look at HoloLens hardware
        1. Inside the HoloLens device
          1. Cameras and sensors
          2. The Lenses
          3. Speakers
          4. Microphones
          5. The processing units
          6. Inertial Measurement Unit
          7. Vent
          8. Connectivity
          9. Control buttons
          10. LED indicator
        2. Putting things together
        3. Turning your device on/off
      2. Cortana on HoloLens
      3. The Mixed Reality world - HoloLens shell
      4. Holograms in reality
        1. HoloLens, holograms and your real world
        2. Hologram and the Windows Holographic platform
        3. Recording holograms - Mixed Reality capture
      5. HoloLens interaction and application model
        1. Interacting with holograms
          1. Gaze
          2. Gesture
          3. Voice
          4. Spatial Audio
          5. Spatial Mapping
        2. HoloLens app life cycles
      6. HoloLens Clicker
      7. Controlling your HoloLens remotely
      8. Key differentiators
        1. Immersive with surroundings
        2. Independent standalone device
        3. No screen staring
        4. Mixed not virtual
        5. Broad scope
      9. Exploring the possibilities with HoloLens
        1. Virtual collaboration
        2. Remote expert guidance
        3. People-centric applications
      10. Development process and team building
        1. Building up the team
          1. Designers
          2. 3D Artists
          3. Developers
          4. Audio designers
          5. Leads
        2. Development process overview
          1. Requirement Envisioning
          2. Storyboarding
          3. Planning, Designing and Prototyping
            1. Planning and Design
            2. Prototyping
          4. Develop
          5. Deploy and Test
      11. Summary
    4. Explore HoloLens as Hologram - Scenario Identification and Sketching
      1. Envisioning
        1. Training scenario for HoloLens
          1. Features
          2. Collaboration
      2. Scenario prioritization
        1. Scenario elaboration
          1. Primary View
          2. Lenses
          3. Speakers
          4. Cameras
          5. Volume control buttons
          6. Brightness control buttons
      3. Sketching the scenarios
        1. Sketching - interaction with speakers
      4. Assets design and development
        1. 3D Modeling
        2. 3D Modeling tools
        3. 3D Modeling workflow
        4. Build your 3D Model
          1. Model sheet - base reference images
          2. Scene creation
          3. Importing model sheet
            1. Import side image
            2. Import front image
            3. Import top image
          4. 3D Model construction with cube
            1. Edge loop
            2. Vertex adjustment
            3. Mirroring
        5. Texturing workflow
          1. UV Unwrapping of model
          2. Create texture in Photoshop
        6. Export as Filmbox (FBX)
        7. Assembling assets in Unity3D
      5. Things to consider while using the 3D Asset Pipeline
        1. Maintaining quads
        2. Checking Surface Normals
        3. Maintaining grouping
        4. Optimizing the scene file
      6. Setting up a development environment
        1. Prerequisites
        2. Installation checklist
      7. Using HoloLens emulator
      8. Summary
    5. Explore HoloLens as Hologram - Developing Application and Deploying on Device
      1. Getting started - creating a new project
        1. The Unity3D basics
          1. The Core interface
            1. Scene View
            2. Game View
            3. Object Hierarchy
            4. Project Assets
            5. Inspector
          2. The Toolbar
          3. The Core concepts
            1. Game Object
            2. Components
            3. Prefabs
            4. Tags
            5. Coordinate system
      2. Setting up a project for holographic application
        1. Setting up the HoloToolkit - Unity
          1. Get The HoloToolkit - Unity
          2. Preparing the package
          3. Import the package
        2. Applying project settings
          1. Behind the scenes
        3. Updating the initial scene
        4. Importing 3D Assets
          1. Exploring the 3D Object
        5. Saving the scene
      3. Build, run, and test the initial setup
      4. Giving life to object - scripting
        1. Highlighting the objects - enabling Gaze
          1. The Gaze manager
          2. The Cursor
          3. Adding the Box Collider
          4. See Gaze in action
          5. Behind the scene
        2. Adding actions to objects - applying gesture
          1. Adding Air Tap on Lenses
          2. Air Tap on Lenses - see it in action
          3. Adding Air Tap on speaker
          4. Air Tap in speaker - see it in action
          5. Behind the scenes
        3. Interaction using Voice Command
          1. Enabling the Microphone capabilities
          2. The Keyword Manager
            1. Defining Keywords
            2. Implementing actions for Keywords
            3. Mapping with Keyword Manager, Keyword and actions
          3. Voice Command - see it in action
          4. Behind the scenes
        4. Interaction using text to speech
          1. Setting up text to speech Manager
          2. Integration of text to speech
            1. Text to speech in application launch
            2. Text to speech in Air Tap
          3. Text to speech - see it in action
          4. Behind the scenes
      5. Deploying your app
        1. Deploy using Visual Studio
        2. Deploy using Build Automation Window
        3. Deploy using Device Portal
          1. Load the package
          2. Start the app
        4. Deploy using Unity remoting
          1. Configure HoloLens device for remoting
          2. Holographic emulation - remoting from Unity
        5. Running the app on a device
      6. Testing your application
        1. Lighting conditions
        2. Space
        3. Angle of views
        4. View distance
        5. Audio or visual hints
        6. Ambient sounds
      7. Summary
    6. Remote Monitoring of Smart Building(s) Using HoloLens - Scenario Identification and Sketching
      1. Envisioning
        1. Solution scope overview
          1. Devices and Sensors
          2. Event Processing
          3. Visualization
            1. Real-time monitoring
            2. Analyse and Act
      2. Scenario prioritization
        1. Scenario elaboration
          1. Internet of Things - Remote Monitoring
            1. Devices
            2. Cloud Gateway (Azure IoT Hub)
            3. Steam Processing (Azure Stream Analytics)
            4. Data Store (Azure Storage)
            5. Machine Learning
          2. Real-time visualization through HoloLens
            1. APIs
            2. Holographic application
      3. Sketching the scenarios
        1. Sketching - Interaction with Rooms and Sensors
      4. 3D Model - structure and components
        1. 3D Model of building
        2. 3D Model of sensors
      5. Summary
    7. Remote Monitoring of Smart Building(s) Using HoloLens - Developing Application and Deploying on Device
      1. Solution development
        1. Logical design diagram
        2. Setting up the backend Infrastructure
          1. Create the Azure Event Hub
          2. Create the Azure Cosmos DB
          3. Create the Stream Analytics Job
        3. Building up the Azure IoT solution
          1. Connecting Device with Azure Event Hub - simulator
          2. Developing the Web API
        4. Setting up a project for a holographic application
          1. Getting started - creating a new project
          2. Import the package
          3. Applying Project Settings
          4. Updating the initial scene
          5. Importing 3D Assets - building model
            1. Exploring the building's 3D Objects
          6. Save the scene
          7. Build, run and test the initial setup
          8. Tracking building floors from 3D Model
          9. Adding Gaze
          10. Adding the cursor
          11. Adding the Box Collider
          12. Add tag to each floor for mapping
          13. Tracking Gaze for each floor
          14. See Gaze in action
            1. Verify the Gazed floor
        5. Connecting a holographic app with Azure
          1. Defining data models
          2. Enabling the InternetClient capabilities
          3. Building Azure Bridge
            1. Getting the data
            2. Attaching the Azure Bridge
          4. Binding data with sensor indicators
            1. See the live data in action
          5. Enabling sensor binding for all floors
        6. Adding actions to floors - applying gesture
          1. Adding Air Tap on floors
          2. Displaying detailed information
            1. Designing the Panel layouts
            2. Grouping UI elements - adding Canvas Group
            3. Binding Data with Panel
            4. Invoking the Panel
            5. Closing the Panel
          3. Air Tap on Lenses - see it in action
      2. Deploying your app onto a device
      3. Extending the current solution
        1. Adding Voice Command
        2. 3D Model placement as table top mode
        3. Additional interaction with services
      4. Summary
    8. Build End-to-End Retail Solution - Scenario Identification and Sketching
      1. Envisioning
        1. Scenario elaboration
          1. Catalog and merchandising
          2. Visualization
          3. Order placement
      2. Sketching the scenarios
        1. Sketching - interaction for catalog and items
        2. Sketching - interaction with 3D Model
      3. 3D Model
      4. Summary
    9. Build End-to-End Retail Scenario - Developing Application and Deploying on Device
      1. Getting the 3D Assets ready
        1. Unity Project and Asset Bundle setup
        2. Importing 3D retail assets
        3. Assets Bundles generation
      2. Solution development
        1. Logical design diagram
        2. Building the backend Azure Solution
          1. Creating the Azure Blob Storage
          2. Creating the Azure Cosmos DB
          3. Developing the Web API
        3. Setting up the holographic project
          1. Importing the main menu 3D Model
            1. Exploring the main menu
          2. Building , running and testing the initial solution
        4. Making the main menu Intractable
          1. Enabling Gaze with cursor
          2. Attaching the script with menu items
          3. Enabling Air Tap with menu items
        5. API connector - connecting with Web API
          1. Defining the application data model
          2. Enabling the IntenetClient capabilities
          3. API Connector
            1. Getting the data
            2. Attaching the API connector
        6. Getting holograms on the fly
          1. Handling the Air Tap on menu item
          2. Downloading assets and rendering holograms on the fly
        7. Associating toolbars with holograms
          1. The toolbar controls
          2. Instantiating toolbars on the fly
            1. Showing controls based on types
        8. Dynamic holograms in action
        9. Adding action to the toolbar controls
          1. Remove product
          2. Add to Cart
          3. Changing pictures for Wall Paints
        10. Placing the holograms - spatial mapping
      3. Deploying your app on a device
      4. Extending the current solution
        1. Dynamic menu for products
        2. Adding Voice Command
        3. Application and storage access security
        4. Rotating the individual product
      5. Summary
    10. Possibilities
      1. HoloLens and Microsoft Bot Framework
        1. Solution
          1. Client authentication
          2. Start conversation
          3. Send message
          4. Receive message
      2. HoloLens in healthcare
        1. Solution
          1. Search data from HealthVault
      3. HoloLens and live streaming
        1. Solution
          1. Setting up media services for live streaming
          2. Consume live streaming within holographic application
      4. Handling secure keys with Azure Key Vault
      5. Summary
    11. Microsoft HoloLens in Enterprise
      1. Microsoft HoloLens Commercial Suite
      2. HoloLens device management using Microsoft Intune
        1. Manual registration
        2. Automatic registration
      3. Installing applications on HoloLens
        1. Device Portal for application deployment
        2. Business Store for application deployment
      4. Summary

    Product information

    • Title: HoloLens Blueprints
    • Author(s): Abhijit Jana, Manish Sharma, Mallikarjuna Rao
    • Release date: June 2017
    • Publisher(s): Packt Publishing
    • ISBN: 9781787281943