2 MODELING OF THE TRACKING AND GRASPING SYSTEM 117
reference point is used to represent the translational motion of the object, the tracking can
be designed with respect to the 3-D point. However, no attempt is made to recover 3-D
feature points of the object on the basis of stereo matching. Instead, only the image of the
reference point in each camera is needed, and it is determined from the 2-D features of the
object within the same camera. By properly defining an error function between the image of
the robot gripper and that of the reference point, the image-based tracking control law is
obtained using the nonlinear regulator theory.
The chapter is organized as follows. Section 2 presents a complete modeling study of the
multicamera hand-eye system. Section 3 proposes a new approach to the estimation of
motion of the object. The problem of determining the reference point in the image planes is
completely studied. A general framework for image-based robot tracking and grasping of a
moving object is presented in Section 4. In Section 5 we report simulation results. Section 6
concludes the chapter.
2 MODELING OF THE TRACKING AND GRASPING SYSTEM
2.1 System Configuration
Figure 4.1 illustrates the configuration of the tracking and grasping system, where J (J ~> 2)
fixed cameras are directed toward the work space of a robot manipulator. The robot's base
coordinate frame is chosen as the world coordinate frame (WCF) of the system, which is
denoted by
OwXwYwZw .
For i= l,..., J, let
OciXciYciZci
denote the ith camera coordinate
frame (CCFi), where Oci and
oc~Zc~
are respectively the center of the lens and the optical axis
of the ith camera. For any point p in the work space, its coordinates in the WCF and CCFi
are denoted respectively by (x~, y~, z~) and (xc p, yc p, zc~) (i = 1,..., J).
The coordinates of point p in the WCF and the CCFi can be related by
Zc d L
(i = 1,..., J) (4.1)
Ycl
YcI
9 Yw
FIGURE 4.1
Configuration of the tracking and grasping system.
1 18 CHAPTER 4 / VISUALLY GUIDED TRACKING AND MANIPULATION
where
r V]
?'il 1"i2 ?'i3 dil
[_Ti7
ri8 ri9 di3
are known 3 x 3 orthonormal and 3 x 1 matrices, respectively.
2.2 Model of the Robot Manipulator
For the task of tracking and grasping a moving object, a six degree-of-freedom PUMA 560
manipulator is used with a parallel jaw gripper mounted on the end effector. The dynamics
of the robot is described by
D(q)q" + C(q, O) + G(q) = -c
(4.2)
where q
= Eql q2 q3
q4 q5
q6] T, 77 = E-c I
"~2 T3 T4 T5
T6] T'
qi, Oi, (1"i
(i = 1, 2 .... ,6) are the
position, velocity, and acceleration of joint i, respectively; r~ is the torque acting at joint i;
and D, C, and G are respectively the 6 x 6 inertia matrix, 6 x 1 vector of centripetal and
Coriolis terms, and 6 x 1 vector of gravity terms. The position and orientation of the gripper,
denoted by [x~ y~, z~] v and [n o s o a~ respectively, can be represented as trigonometric
functions of q by the forward kinematic equations. Alternatively, using the OAT representa-
tion [14], the orientation of the gripper can also be described by three independent Euler
angles O ~ A y, and T ~ Clearly, O.0, A y,
T 9 are
also functions of the joint displacement q.
On introducing state variables X l q, x2 //, and x v [Xl v v = = = x2], (4.2) has the form
I x2 ] E 0 ]
2 2 D- l(C(x) + G(xl)) + D-
I(X1) T
(4.3)
~- f(x) +
g(x): (4.4)
The pose (position and orientation) of the gripper can be described by
g
Xw
g
Zw
0 o
A o
T o
_ _
h4(x l ) I
hs(x
1) I
_h6(x
1)]
(4.5)
It is well known [15] that for system (4.4) with output equation (4.5), a state feedback
-c = c~(x) + fl(x)v
(4.6)
and change of coordinates
[~1 ~2 "'" ~:2] m = ~(x) (4.7)

Get Control in Robotics and Automation now with O’Reilly online learning.

O’Reilly members experience live online training, plus books, videos, and digital content from 200+ publishers.