3D Shape Analysis

13Cross‐domain Retrieval

13.1 Introduction

In Chapter 12, we emphasized on the need for efficient 3D search tools and presented the most important techniques for 3D shape retrieval using 3D models as queries. Ideally, however, a search engine should provide a mechanism that enables users to retrieve relevant information, whether it is an image, a 3D model, or a 2D sketch, using queries which can also be images, sketches, text, or 3D models. For instance, the user may draw a 2D sketch depicting an airplane and queries a collection of images looking for related photos. Other users may want to search for sketches that are similar to a given photo or 3D model. This is referred to as cross‐domain retrieval. It aims at providing the users with an efficient way to search for shapes independently of their representations.

The main difficulty with cross‐domain retrieval is that different modalities lie in different (disjoint) spaces or domains. The challenge is then how to narrow the modality gap between these different types of representations. Early works that attempted to solve this problem focused solely on two modalities, e.g. photos vs. 3D shapes, or hand‐drawn sketches vs. 3D shapes. For instance, in photo‐based 3D shape retrieval, the signature of the photo is compared to the signatures of the 2D projections of the 3D model, see view‐based 3D shape descriptors described in Section 4.3. Similarly, sketch‐based 3D shape retrieval methods extract 2D silhouettes from the 3D models ...