Chapter 13An AI-Powered 3D App
Having learned about HoloLens specific functionalities, let’s continue to work with the Mixed Reality Toolkit for Unity (MRTKu). In this chapter you will create a new app, VisionAssistant3D. (See Figure 13-1.) In doing so you will learn how to capture photos from the camera on the UWP platform (including the HoloLens world-facing camera) using Unity’s PhotoCapture object. You will then learn how to configure the app to display the camera image in a Quad object and utilize artificial intelligence (AI) to describe the camera image. To do so you will send the image to Microsoft Cognitive Services (MCS) for analysis. As in Chapter 6, you will use the Computer Vision API to obtain a text description of the objects in ...
Get Programming for Mixed Reality with Windows 10, Unity, Vuforia and UrhoSharp, First Edition now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
