This chapter presents the application of a reinforcement learning algorithm to learning online how to manipulate micro-objects. What makes this application original is that the action policy has been learned not thanks to a simulator but by controlling the real process. This work related to reinforcement learning for micro-robotics has been conducted at the Automatic control and Micro-Mechatronic Systems department of the FEMTO-ST Institute (Besançon, France). It was first described in [ADD 05].
Micro-robotics has the general objective of designing, realizing and controlling compact robotic systems used to manipulate objects whose dimensions typically range between one micrometer and one millimeter for various applications (instrumentation, micro-assembly, biomedical applications).
Considering the dimensions and the required precision, micro-robotics faces practical difficulties which are quite different from in classical robotics:
– at the actuators level: micro-robotics employs new, more compact, actuation principles, for example, based on the use of active materials; such actuators are often strongly non-linear;
– at the sensors level: the reduced volume of the applications makes it difficult to set up a sufficient number of sensors; employing a vision system (via a microscope) is often the main mean to observe and measure;
– at the level of the interactions between the robot and the surrounding objects: ...