Microsoft HoloLens By Example

Understanding and using the real world

One of the opportunities (and challenges) with MR is understanding enough of the real world to make it useful in the experiences you create. We have seen how we can make use of our understanding of the surrounding surfaces, but imagine how much more magical the experience would be if we could recognize objects and make appropriate use of them. For example, being able to recognize a chair would give us the ability to let virtual characters sit on the chair or know that a holographic phone (the ones that sit on a desk, if they still exist) is best placed on a desk.

This process of recognizing objects from a digital camera feed sits in the field of computer vision. Luckily, for us, there are a few libraries ...