2 MODELING OF THE TRACKING AND GRASPING SYSTEM 117

reference point is used to represent the translational motion of the object, the tracking can

be designed with respect to the 3-D point. However, no attempt is made to recover 3-D

feature points of the object on the basis of stereo matching. Instead, only the image of the

reference point in each camera is needed, and it is determined from the 2-D features of the

object within the same camera. By properly defining an error function between the image of

the robot gripper and that of the reference point, the image-based tracking control law is

obtained using the nonlinear regulator theory.

The chapter is organized as follows. Section 2 presents a complete modeling study of the

multicamera hand-eye system. Section 3 proposes a new approach to the estimation of

motion of the object. The problem of determining the reference point in the image planes is

completely studied. A general framework for image-based robot tracking and grasping of a

moving object is presented in Section 4. In Section 5 we report simulation results. Section 6

concludes the chapter.

2 MODELING OF THE TRACKING AND GRASPING SYSTEM

2.1 System Configuration

Figure 4.1 illustrates the configuration of the tracking and grasping system, where J (J ~> 2)

fixed cameras are directed toward the work space of a robot manipulator. The robot's base

coordinate frame is chosen as the world coordinate frame (WCF) of the system, which is

denoted by

OwXwYwZw .

For i= l,..., J, let

OciXciYciZci

denote the ith camera coordinate

frame (CCFi), where Oci and

oc~Zc~

are respectively the center of the lens and the optical axis

of the ith camera. For any point p in the work space, its coordinates in the WCF and CCFi

are denoted respectively by (x~, y~, z~) and (xc p, yc p, zc~) (i = 1,..., J).

The coordinates of point p in the WCF and the CCFi can be related by

Zc d L

(i = 1,..., J) (4.1)

Ycl

YcI

9 Yw

FIGURE 4.1

Configuration of the tracking and grasping system.

1 18 CHAPTER 4 / VISUALLY GUIDED TRACKING AND MANIPULATION

where

r V]

?'il 1"i2 ?'i3 dil

[_Ti7

ri8 ri9 di3

are known 3 x 3 orthonormal and 3 x 1 matrices, respectively.

2.2 Model of the Robot Manipulator

For the task of tracking and grasping a moving object, a six degree-of-freedom PUMA 560

manipulator is used with a parallel jaw gripper mounted on the end effector. The dynamics

of the robot is described by

D(q)q" + C(q, O) + G(q) = -c

(4.2)

where q

= Eql q2 q3

q4 q5

q6] T, 77 = E-c I

"~2 T3 T4 T5

T6] T'

qi, Oi, (1"i

(i = 1, 2 .... ,6) are the

position, velocity, and acceleration of joint i, respectively; r~ is the torque acting at joint i;

and D, C, and G are respectively the 6 x 6 inertia matrix, 6 x 1 vector of centripetal and

Coriolis terms, and 6 x 1 vector of gravity terms. The position and orientation of the gripper,

denoted by [x~ y~, z~] v and [n o s o a~ respectively, can be represented as trigonometric

functions of q by the forward kinematic equations. Alternatively, using the OAT representa-

tion [14], the orientation of the gripper can also be described by three independent Euler

angles O ~ A y, and T ~ Clearly, O.0, A y,

T 9 are

also functions of the joint displacement q.

On introducing state variables X l q, x2 //, and x v [Xl v v = = = x2], (4.2) has the form

I x2 ] E 0 ]

2 2 D- l(C(x) + G(xl)) + D-

I(X1) T

(4.3)

~- f(x) +

g(x): (4.4)

The pose (position and orientation) of the gripper can be described by

g

Xw

g

Zw

0 o

A o

T o

_ _

h4(x l ) I

hs(x

1) I

_h6(x

1)]

(4.5)

It is well known [15] that for system (4.4) with output equation (4.5), a state feedback

-c = c~(x) + fl(x)v

(4.6)

and change of coordinates

[~1 ~2 "'" ~:2] m = ~(x) (4.7)

Get *Control in Robotics and Automation* now with O’Reilly online learning.

O’Reilly members experience live online training, plus books, videos, and digital content from 200+ publishers.