Picking objects in 3D space is a common problem. If you want your 3D simulation to interact with a mouse, we need to solve this problem. To implement picking, we need to find the pixel that the user has clicked relative to both the near and far planes of the camera. We can construct a ray from the point on the near plane to the point on the far plane. Finally, we can query the world using this ray.

The job of a graphics pipeline is to take a 3D point in world space and project it onto the screen. This transformation from world space to screen space is called Projection. To find the 3D world space position of a point based on the 2D pixel position of that same point we need to do the opposite of what the graphics pipeline does. Putting a ...

Get Game Physics Cookbook now with O’Reilly online learning.

O’Reilly members experience live online training, plus books, videos, and digital content from 200+ publishers.