of the minimization algorithm, gains for energy function, number of control points). When
the key factors have been identified, future work will focus on improving the critical elements
of the system.
This work draws on two streams of research in the computer vision and robotics communi-
ties. We have combined techniques developed by the visual servoing community with contour
extraction techniques developed in the graphics and computer vision communities.
Several research efforts have focused on using vision information in the dynamic feedback
loop [7, 10, 13, 14, 19, 24, 27]. Weiss
et al.
[24] have proposed a model reference adaptive
control scheme for robotic visual servoing. In this work, servoing is performed with the goal
of reducing the error between the desired image attributes (center of mass, first or second
moment of the image) and the current image attributes. The verification of the proposed
algorithms has been limited to simulation. Allen
et al.
[1] have proposed an approach that
uses image-differencing techniques in order to track and grab a moving object. Dickmanns
[11, 12] has presented methods (Kalman filters) for the integration of vision information in
the feedback loop of various mechanical systems such as satellites and automobiles. Koivo
and Houshangi [17] have proposed an adaptive scheme for visually servoing a manipulator
based on the information obtained by a static sensor. Feddema and Lee [13] have proposed
a MIMO adaptive controller for hand-eye visual tracking. Their work has been used as the
basis for our approach. Several other researchers [3, 18] have proposed strategies for
vision-based exploration. Finally, Ghosh [14] has addressed several vision-based robotic
issues with the aid of new "Realization Theory" for perspective systems.
The concept of active deformable models, also called "snakes," was first introduced to the
field of computer vision by Kass
et al.
[16]. Snakes have been used in a number of
applications including image-based tracking of rigid and nonrigid objects. Using snakes
requires a minimization process of an energy function. Several techniques have been used to
solve this problem, including variational calculus [16], dynamic programming [2], and
greedy methods using heuristics [28, 29]. The latter method has the advantage of being fast
as well as numerically stable. Our method uses a greedy method similar to that used by
Williams and Shah [29] and Yoshimi and Allen [-28].
Other researchers have also combined elements of visual servoing and active deformable
model techniques to approach different problems than the one presented in this chapter.
Blake, Curwen, and Zisserman [4] have presented a different algorithm for contour
estimation and used it in a system that tracks a contour in an image (it does not include a
robotic component). Yoshimi and Allen [28] have used a greedy, iterative minimization
algorithm to track a robotic finger with a static camera and detect contact between the finger
and a stationary object. Finally, Colombo
et al.
[9] published a description of a system that
uses a spline contour model to plan and execute a movement that positions an eye-in-hand
robot so as to bring a known object into a canonical orientation relative to the camera. They
report initial simulation results.
We describe a system that tracks a moving, deformable object in the work space of a robotic
arm with an eye-in-hand camera. For these experiments, we have used a figure-ground
approach to object detection and identification. The figure-ground methodology allows

Get Control in Robotics and Automation now with O’Reilly online learning.

O’Reilly members experience live online training, plus books, videos, and digital content from 200+ publishers.