Cameras, laser scanners and vision software are all allowing Fraunhofer IPA’s new customer service robot, Care-O-bot 4, to greet shoppers and show them around an electronics store in Germany.
The imaging technology enables the robot, alias Paul, to observe and respond to the mood of customers through observing their gestures and facial expressions. Paul has been greeting customers at electronics retailer Saturn's store in Ingolstadt, Germany since the end of October.
Paul, a member of the Care-O-bot 4 robot family, has been greeting customers in Saturn-Markt Ingolstadt since the end of October 2016. (Credit: Saturn)
For the Saturn assignment, which will run initially for six months, the Stuttgart-based Fraunhofer IPA team working on Care-O-bot 4 supplemented its software and hardware with additional features. ‘For example, we specified Paul’s navigation, dialogue communication and facial expressions,’ explained Ulrich Reiser, Fraunhofer IPA project manager.
A set of laser scanners implemented at shin level monitor the robot’s orientation, allowing it to navigate the store effectively. In addition, its front-facing camera, together with Fraunhofer IIS’ Shore software, enable it to ascertain the emotional state of the customer. The robot is then able to communicate with the customer through a system of microphones and cameras for speech and gesture recognition. If they ask it about a certain product, it accompanies the customer to the department and points them in the direction of the relevant shelf.
Fraunhofer IPA is now working with its spin-off company, Unity Robotics, to ensure the solution is suitable for use in various commercial environments. Saturn is the first retailer to make use of the robot platform in a real-life scenario. ‘With Paul, we are offering our customers the opportunity to get to know one of the most advanced robots in the world,’ said Martin Wild, chief digital officer at Media-Saturn-Holding.
The robot has a modular construction and an open software interface, meaning it can be used in many different application areas. It could, for example, conceivably be used as a mobile information kiosk in shops and airports, for pick-up and delivery services in homes and offices, for security applications, or as an attraction in a robotics museum.
‘We retain an interest in ensuring that as many scientists and companies as possible are able to work with the system in order to expand its potential areas of application,’ Reiser of Fraunhofer IPA explained. Within the framework of the Care-O-bot 4 research platform, experts from around the world can further develop the software and hardware components. Together with Unity Robotics, Fraunhofer IPA experts are intending to adapt applications to the needs of different companies, making Care-O-bot 4 available as a commercial product.
