214 CHAPTER 6 / FEEDBACK CONTROL WITH FORCE AND VISUAL SENSOR FUSION
robotic manipulation tasks. By monitoring the resolvability of the two sensing modes with
respect to the task, the information provided by the disparate sensors can be seamlessly
assimilated during task execution. A nonlinear force-vision servoing algorithm that uses
force and vision resolvability to switch between sensing modes demonstrates the advantages
of the assimilation technique. Contact transitions between a stiff manipulator and rigid
environment, a system configuration that easily becomes unstable when force control alone
is used, are robustly achieved. Experimental results show that the nonlinear controller is able
to satisfy simultaneously the conflicting task requirements of fast approach velocities,
maintaining stability, minimizing impact forces, and suppressing bounce between contact
surfaces. The proper assimilation of force and vision feedback is the key to the success of this
 R. C. Smith and P. Cheeseman. On the representation and estimation of spatial uncertainty. Int. J. Robot. Res.
 H. F. Durrant-Whyte. hTtegration. Coordination and Control of Multi-Sensor Robot Systems. Kluwer Academic
Publishers, Boston, 1988.
 J. M. Richardson and K. A. Marsh. Fusion of multisensor data. Int. J. Robot. Res. 7(6):78-96, 1988.
 G. D. Hager. Task-Directed Sensor Fusion and Planning--A Computational Approach, Kluwer Academic
 R. Jain. Environment models and information assimilation. Tech. Rept. RJ 6866 (65692), IBM, Yorktown
Heights, NY, 1989.
 B. Nelson and P. K. Khosla. Integrating sensor placement and visual tracking strategies. In Experimental
Robotics HI. The Third hTternational Symposium, Kyoto, October 28 -30, 1993, eds. T. Yoshikawa and F.
Miyazaki. Springer-Verlag, London, 1994, pp. 169-181.
 B. J. Nelson and P. K. Khosla. The resolvability ellipsoid for visual servoing. Proceedings of the 1994
Conference on Computer Vision and Pattern Recognition ( CVPR94), 1994, pp. 829-832.
 B.J. Nelson and P. K. Khosla. Visually servoed manipulation using an active camera. Proceedings Thirty-Third
Annual Alh, rton Conference on (~kmmmnication, Control, and Comput#1,~. University of Illinois at Urbana-
Champaign, Oct. 4- 6, 1995.
 D. E. Whitney. Historical perspective and state of the art on robot force control. Proceedings 1985 International
Conference on Robotics and Automation. IEEE, New York, 1985, pp. 262 268.
 N. Hogan. Impedance control: An approach to manipulation, Parts I-Ill. ASME J. Dyn. Syst. Meas. Control
 M. H. Raibert and J. J. Craig. llybrid position/force control of manipulators. ASME J. Dyn. Syst. Meas.
Control 103(2):126-133, 1981.
 T. Yoshikawa. Dynamic hybrid position/force control of robot manipulators .... description of hand constraints
and calculation of joint driving force. IEEE J. Robot. Automat. 3(5):386-392, 1987.
 R. A. Volpe and P. K. Khosla. Experimental verification of a strategy for impact control. Proceedings 1991
IEEE International Conference on Robotics and Automation. IEEE, New York, 1991, pp. 1854-1860.
 C. An and J. Hollerbach. Dynamic stability issues in force control of manipulators. Proceedings 1987
hTternational Conference on Robotics and Automation. IEEE, New York, 1987, pp. 860-896.
 S. Eppinger and W. Seering. Understanding bandwidth limitations on robot force control. Proceedings 1987
International Conference on Robotics and Automation. IEEE, New York, 1987, pp. 904-909.
 J. M. Hyde and M. R. Cutkosky. Contact transition: an experimental study. Proceedings 1993 International
Conference on Robotics and Automation. IEEE, New York, 1993, pp. 363-368.
 N. Hogan. Stable execution of contact tasks using impedance control. Proceedings 1987 International
Conference on Robotics and Automation. IEEE, New York, 1987, pp. 1047-1054.
 H. Kazerooni. Robust, non-linear impedance control for robot manipulators. Proceedings 1987 International
Conference on Robotics and Automation. IEEE, New York, 1987, pp. 741-750.
 O. Khatib and J. Burdick. Motion and force control for robot manipulators. Proceedings 1986 International
Conference on Robotics and Automation. IEEE, New York, 1986, pp. 1381-1386.
 H. P. Qian and J. De Schutter. Introducing active linear and nonlinear damping to enable stable high gain