314 CHAPTER 11 / AUTOMATED INTEGRATION OF MULTIPLE SENSORS
the rationalization that the results can be easily adapted to real images. Although this may
well be true theoretically, often the problems and difficulties inherent in processing real sensor
data are such that simulation-based techniques resolve few of the real problems. That is, the
distortions, sensor anomalies, noise, and so on inherent in real images so dominate the
processing problem and either the simulation-based technique is solving such a trivial
component or the anticipated mapping function from real images to simulation quality
images is at least as complex as the original IP problem. It is our view, therefore, that real
sensor data must be the focus from day one for all but function encoding verification,
graphical explanation, or preliminary concept generation. The techniques presented in this
chapter are developed and evaluated on actual multimodal sensor data consistent with
real-world robotic applications, that is, a laser range camera used for robotic navigation and
ground conductivity data used for buried waste assessment.
The following section describes the traditional approach and provides a comparative
overview of our adaptive learning approach to MSI. This approach is detailed in Section 3.
Section 4 describes the sensory target domains, including a description of the physical nature
of each sensor system (laser range camera, LRC, and ground conductivity, GC, sensor) and
their targeting environments. Section 5 presents a specific sensor anomaly common to LRCs
and our MSI approach to resolving its distortions, complete with empirical results. The sixth
section presents an empirical evalution of the performance of our automated MSI system
applied to LRC images. Section 6 validates these results empirically in the learned (LRC) and
unlearned (GC) domains. A summary section follows.
Although MSI has a generally accepted overall purpose, that is, to generate a more reliable
and consistent representation, the
goals of each MSI implementation vary from one
application to another [-4, 5, 7]. MSI techniques combine multimodal sensor images (e.g.,
sonar and visual , distance and reflectance [-1, 13]), multiple single modal sensor images
(e.g., multisampling, a single sensor's images displaced in time or in space [-14]), or com-
binations of these.
MSI is necessary because of basic ambiguities inherent in our current sensor imaging
technologies. Sensor ambiguities derive from two basic causes: limitations of the physical
attribute being measured by the sensor (e.g., visible light's inability to permeate opaque
surfaces, sonar edge effects) and the sensor's inaccuracies in making measurements (e.g., noise,
resolution). Ambiguity exists as long as the mapping from reality to image is not one to one.
That is, if different "realities" lead to identical images, a single image cannot reveal the
particular reality that was the truth; for example, a two-dimensional (2-D) visual image of
an opaque object cannot reveal its interior or its hidden surfaces, hence an infinite number
of "realities" would result in the same sensor image.
MSI techniques attempt to resolve some of these ambiguities by appropriately coupling
complementary images to eliminate possible inverse mappings. What constitutes the most
advantageous MSI technique is dependent on the given application domain, available
sensors, and task requirements. Multisampling is perhaps the simplest MSI technique and is
used primarily to reduce noise. Merging multiple single modal images  displaced in space
can improve three-dimensional information, reveal otherwise occluded areas, and reduce
orientation-induced artifacts, such as glare or sonar positional anomalities. Merging multiple
single modal images displaced in time can be used to reduce temporal effects or to isolate
them. Employing multimodal sensors permits exploitation of each sensor's strengths without