2 MODELING OF THE TRACKING AND GRASPING SYSTEM 117
reference point is used to represent the translational motion of the object, the tracking can
be designed with respect to the 3-D point. However, no attempt is made to recover 3-D
feature points of the object on the basis of stereo matching. Instead, only the image of the
reference point in each camera is needed, and it is determined from the 2-D features of the
object within the same camera. By properly defining an error function between the image of
the robot gripper and that of the reference point, the image-based tracking control law is
obtained using the nonlinear regulator theory.
The chapter is organized as follows. Section 2 presents a complete modeling study of the
multicamera hand-eye system. Section 3 proposes a new approach to the estimation of
motion of the object. The problem of determining the reference point in the image planes is
completely studied. A general framework for image-based robot tracking and grasping of a
moving object is presented in Section 4. In Section 5 we report simulation results. Section 6
concludes the chapter.
2 MODELING OF THE TRACKING AND GRASPING SYSTEM
2.1 System Configuration
Figure 4.1 illustrates the configuration of the tracking and grasping system, where J (J ~> 2)
fixed cameras are directed toward the work space of a robot manipulator. The robot's base
coordinate frame is chosen as the world coordinate frame (WCF) of the system, which is
denoted by
OwXwYwZw .
For i= l,..., J, let
OciXciYciZci
denote the ith camera coordinate
frame (CCFi), where Oci and
oc~Zc~
are respectively the center of the lens and the optical axis
of the ith camera. For any point p in the work space, its coordinates in the WCF and CCFi
are denoted respectively by (x~, y~, z~) and (xc p, yc p, zc~) (i = 1,..., J).
The coordinates of point p in the WCF and the CCFi can be related by
Zc d L
(i = 1,..., J) (4.1)
Ycl
YcI
9 Yw
FIGURE 4.1
Configuration of the tracking and grasping system.
1 18 CHAPTER 4 / VISUALLY GUIDED TRACKING AND MANIPULATION
where
r V]
?'il 1"i2 ?'i3 dil
[_Ti7
ri8 ri9 di3
are known 3 x 3 orthonormal and 3 x 1 matrices, respectively.
2.2 Model of the Robot Manipulator
For the task of tracking and grasping a moving object, a six degree-of-freedom PUMA 560
manipulator is used with a parallel jaw gripper mounted on the end effector. The dynamics
of the robot is described by
D(q)q" + C(q, O) + G(q) = -c
(4.2)
where q
= Eql q2 q3
q4 q5
q6] T, 77 = E-c I
"~2 T3 T4 T5
T6] T'
qi, Oi, (1"i
(i = 1, 2 .... ,6) are the
position, velocity, and acceleration of joint i, respectively; r~ is the torque acting at joint i;
and D, C, and G are respectively the 6 x 6 inertia matrix, 6 x 1 vector of centripetal and
Coriolis terms, and 6 x 1 vector of gravity terms. The position and orientation of the gripper,
denoted by [x~ y~, z~] v and [n o s o a~ respectively, can be represented as trigonometric
functions of q by the forward kinematic equations. Alternatively, using the OAT representa-
tion [14], the orientation of the gripper can also be described by three independent Euler
angles O ~ A y, and T ~ Clearly, O.0, A y,
T 9 are
also functions of the joint displacement q.
On introducing state variables X l q, x2 //, and x v [Xl v v = = = x2], (4.2) has the form
I x2 ] E 0 ]
2 2 D- l(C(x) + G(xl)) + D-
I(X1) T
(4.3)
~- f(x) +
g(x): (4.4)
The pose (position and orientation) of the gripper can be described by
g
Xw
g
Zw
0 o
A o
T o
_ _
h4(x l ) I
hs(x
1) I
_h6(x
1)]
(4.5)
It is well known [15] that for system (4.4) with output equation (4.5), a state feedback
-c = c~(x) + fl(x)v
(4.6)
and change of coordinates
[~1 ~2 "'" ~:2] m = ~(x) (4.7)
Get Control in Robotics and Automation now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.