Show Details

Highly Observant Robots

April 4, 2013

New robots are learning how to handle objects by watching humans.



Highly observant robots. I’m Bob Hirshon and this is Science Update.

A good assistant can learn a lot about a job just by watching.  Now, computer scientists at Cornell University are designing robots that can do the same thing.  Team leader Ashutosh Saxena says their robots can analyze how people use an object, like a plate or a water pitcher, and learn to handle it in similar ways. The robots also learn to apply rules based on the object’s function.

ASHUTOSH SAXENA (Cornell University):

For example, if something contains a liquid, it would not tilt it when moving it around.

The robots track people’s movements with Microsoft Kinect, a motion sensor used in home games. It also detects the objects being used, and compares its observations with a database of objects, their functions, and what people do with them. Saxena says the robots not only learn to imitate tasks, but also to anticipate the next step – for instance, by clearing empty dishes after someone finishes eating. I’m Bob Hirshon for AAAS, the Science Society.

After watching a person making cereal (left), this robot knows it's time to put the milk away (right). (Saxena Lab/Cornell University)