Engineers at the Politecnico di Milano in Italy are developing a vision system that can infer what a human is going to do in order for a robot to operate more effectively with its human co-workers.
Professor Paolo Rocco at the Politecnico di Milano gave a presentation at the European Machine Vision Forum in Bologna, Italy about the research his group is conducting on cobots, robots designed to work with humans. The conference took place from 5 to 7 September.
One of the projects Rocco spoke about is a vision system that can predict a human’s intention by looking at how they're working. In an assembly process, for instance, once the robot knows what part the human will pick up, it can move to another task or potentially aid the human.
The system uses a deep learning algorithm trained, in the example Rocco showed, by tracking the motion of the human's right hand. In this example the engineer was assembling a box by adding a lid to it. The human and robot are in front of one another and between them are two trays: one with the box, the other with the lid. Whenever the human goes to take a box or pick up a lid, the robot understands and does the complementary action. So, when the human takes the lid, the robot picks up the box and positions it for the human to attach the lid, and vice versa.
‘Humans normally do a sequence of actions,’ Rocco explained during his presentation. ‘The most recent thing we’ve done is to predict the sequence of actions the human will do. So, whenever the human wants to make a collaborative action, the robot will be ready, so there will be no waste of time.’
In Rocco’s setup, the robot constantly monitors the human’s action so that the cycle of the human and that of the robot synchronises. ‘The benefit was a 20 per cent reduction in cycle time just using the machine learning algorithm to infer what the human is doing,’ Rocco said.
Rocco’s group has recently founded a spin-off company called Smart Robots. The company makes a smart hardware and software solution, which optimises the space where human operators and robots work side by side. The device can be linked to robots from various manufacturers, and can be used to ensure collision avoidance, or reschedule the robot’s tasks if it interferes with what the human is doing.
