Artificial Intelligence for Robotics

Google’s SAC-X

Google is trying a slightly different approach to the robot arm problem. In their SAC-X program, which stands for Scheduled Auxiliary Control, they surmise that it can be quite difficult to assign reward points to individual movements of the robot arm. They break down a complex task into smaller auxiliary tasks, and give reward points for those supporting tasks to let the robot build up to a complicated challenge. If we were stacking blocks with a robot arm, we might separate picking up the block as one task, moving with the block in hand as another, and so on. Google referred to this as a "sparse reward" problem if they only did reinforcement on the main task, stacking a block on on top of another. You can imagine in the process ...