The first step in our process after "begin cleaning" is "look for toys". This storyboard frame is what the robot sees as it is commanded to start cleaning. It sees the room, which has three kinds of objects visible: toys, things that are not toys (the green ottoman and the red fireplace), and the room itself, the walls, and the floor:
We could select any sort of sensor to detect our toys and direct our robot. We could have a Lidar, or a thermal scanner, or sonar. Let's make an hypothesis that the best sensor tool for this task is a regular USB camera. We have control of the lighting, the toys are not particularly ...
