Coding4Fun

Warping the Camera Coordinates

From the previous chapter, you learned how to set up a demo program that reads the data from the Wii remote accelerometers. To get the coordinate values of the Wii remote's IR sensor, you can use the WiimoteState.IRState.IRSensors array inside of your WiimoteChangedEventHandler. You can read the coordinates of each visible dot, but this will only give you the location in the coordinate space of the camera. Unless we have the Wii remote positioned directly in front of our display such that every pixel is accurately aligned, we must transform the raw camera coordinates into sensible display coordinates. Or, more precisely, we must find the correspondence between the camera pixels and the display pixels. To do this, we can use something called a planar homography.

When you look at the world through the lens of a camera, the world is captured as a flat image and everything becomes distorted according to the rules of perspective geometry. Objects that are farther away look smaller and squares transform into trapezoids. When looking at a flat surface, this distortion can be encapsulated by a warping matrix called a planar homography. This is what we need to map our camera coordinates on the display.

Explaining exactly how or why this works is outside the scope of this chapter. It is all part of the magic of linear algebra. But for the sake of using the following code, all you need to know are eight numbers: the corners of a known rectangle on your display, ...