Learning OpenCV 3

Chapter 19. Projection and Three-Dimensional Vision

In this chapter, we’ll move into three-dimensional vision. First, we will investigate three- to two-dimensional projections and the inverse operation (as much as this operation can be inverted), and then move on to multicamera stereo depth perception. To do this, we’ll have to carry along some of the concepts from Chapter 18. We’ll need the camera intrinsics matrix M, the distortion coefficients, the rotation matrix R, the translation vector , and especially the homography matrix H.¹

We’ll start by discussing projection into the three-dimensional world using a calibrated camera and reviewing affine and projective transforms (which we first encountered in Chapter 11); then we’ll move on to an example of how to get a bird’s-eye view of a ground plane.² We’ll also discuss, in a little more detail, cv::solvePnP(), which we first saw in Chapter 18. In this context, we will see how this algorithm can be used to find the three-dimensional pose (position and rotation) of a known three-dimensional object in an image.

With those concepts in hand, we will then move into the three-dimensional geometry and multiple imagers. In general, there is no reliable way to do calibration or to extract three-dimensional information without multiple images. The most obvious case in which we use multiple images to reconstruct a three-dimensional scene is ...