3D Visual Communications

2.3 3D Scene Representation without Geometry – Image-Based Representation

Geometry-based representations use geometry primitives to represent objects in the scene, the content being transmitted from the sender to the receiver, and the receiver synthesizes the views by rasterizing the primitives with computer graphics techniques. However, the need of advanced graphics hardware for rendering and the lack of realism in the synthesized views make these geometry-based methods not appropriate for 3DTV systems. Instead, original TV systems capture the real world in a single-planar-view high-definition video stream and the video can deliver realism to viewers. Thus researchers propose to extend the idea to capture the real world in a set of videos and then synthesize new views from these sets of videos in the receiver side to deliver the reality to the consumer. This idea is called image-based representation. The image-based representations are categorized into:

1. single texture techniques: a single texture can represent the appearance of an object or the entire world by a single and connected surface such as a cylinder or a plane.
2. multiple texture techniques: a multiple texture representation refers to using a set of original camera views to represent a 3D scene.

In the following, we will first give a mathematical description of the process, namely, the plenoptic function, and then we will discuss image-based representations.

2.3.1 Plenoptic Function

3DTV should be able to deliver the ...