Christoph Hellmann Santos, of the ROS Industrial Consortium Europe and Fraunhofer IPA, explains why more industrial users are now building robots through the open source software platform, Robot Operating System
In recent years, robot automation has tapped into the rise of open source software, notably with the success of the Robot Operating System (ROS). ROS gives developers control of a wide range of industrial robots using modern programming languages such as Python or C/C++. It is also great for robot systems incorporating machine vision, as easy integration of different cameras, robots and image processing tools makes it ideal for these types of application.
Currently, more than 140 different robots can be controlled using ROS, including autonomous ground vehicles, drones, agricultural robots or even underwater robots. Developers originally used ROS mainly in research-oriented applications, but currently we see a strong trend towards industrial applications. In addition, thanks to the H2020 project, ROSIN, the quality of the ROS open source software has been brought close to industrial standards.
Why is the open source model becoming so popular? The answer lies partly in Marc Andreessen's famous statement that 'software is eating the world', which continues to be true including in robotics.
Progress in robot automation can only be achieved by creating better, more intelligent software, as robot kinematics have stayed the same for decades. However, creating smarter software is quite challenging and requires complex software engineering as, in addition to designing a completely new automation function, many other problems need to be solved. Without a large software development team, building robots for complex automation problems is close to impossible using conventional methods. One answer to this is to use open source software such as ROS.
ROS is a software platform that provides tools and components for solving most of the problems that are not directly related to an actual new automation function. Thus, it offers a fast track to new technology, by taking advantage of the common, shared R&D that has already been solved around the world. When developing a robot application based on ROS, the engineer can simply reuse the shared code, and focus entirely on putting their own, unique functionality on top. Resources can therefore be focused on value-added work, which gives a competitive advantage.
To pay back, the engineer or organisation contributes to the common open source platform, by providing improvements, bug fixes and new modules that they would like to be developed further by the open source community. ROS, for example, provides intra-process communication, support for all major industrial robots and path planning for industrial robots. Currently, more than 2,000 software packages are available for ROS.
One of the major advantages of using ROS for building robot automation solutions is the ease of integrating machine vision libraries such as OpenCV or Point Cloud Library (PCL). This means robot software developers can create systems that require complex machine vision. There's also the advantage that a vision solution can be switched easily to another type of robot when working with ROS.
A rising number of organisations providing robot solutions for manufacturing have realised this advantage and joined forces in the ROS Industrial Consortium, spreading ROS in industrial applications. Currently, the consortium has approximately 80 members. The open source community around ROS, consisting of developers, researchers and companies, is what makes ROS tick and enables fast development of new software components.
Recognising dirt
The robotics team of Fraunhofer IPA’s robot and assistive systems department uses ROS for industrial machine vision applications for robotics. Currently, the team is working on dirt recognition for mobile cleaning robots, programming the robot to only trigger cleaning when it identifies dirt. This saves energy and detergents, and reduces wear on the cleaning tools.
For its dirt recognition module, the team uses ROS because the ecosystem gives access to advanced vision libraries. In addition, support tools such as visualisation with RVIZ are available out of the box. Moreover, ROS enables the simple integration of the developed dirt detection modules into ROS-based mobile robots. By using bridging mechanisms, integration into robots running other control software is also possible. Metralabs and Bosch Engineering have tested the dirt detection module on their robots.
Richard Bormann, team manager at Fraunhofer IPA, commented: 'We develop using ROS because our modules can be easily integrated into a large number of mobile robots.'
Drag&bot's GUI for building robot applications
Martin Naumann, CEO of German software firm Drag&bot, added his reasons for working with ROS: 'ROS as reliable robotics middleware allows us to offer common interfaces to integrate all kinds of periphery and functionality – such as cameras and vision algorithms – into Drag&bot, and also to profit from new developments of the ROS community.'
Drag&bot offers a software suite designed to simplify and unify programming for industrial robots. Its software enables workers to program robots without special training, and reduces programming time, integration costs and automation project duration.
Drag&bot software provides a graphical user interface for building robot applications using function blocks. To program a robot, the worker only needs to put these function blocks in the right sequence using a drag and drop mechanism, and configure the function blocks' parameters in the graphical user interface.
The software also offers the integration of 2D and 3D vision function blocks, for example for workpiece pose detection or bin picking. Drag&bot relies on ROS to provide customers with a manufacturer-independent robot programming solution, and makes use of ROS' machine vision capabilities.
A third ROS user is the Norwegian robot systems integrator, PPM Robotics. PPM provides industrial robots for grinding, deburring, polishing and welding, and is the main developer of Nachi's program interface FlexGui 4.0, which is also released as open source for ROS.
PPM has built ROSweld, which is the first heavy robot welding system using machine learning in the welding process modelling for planning multipass welding. ROSweld also includes planning from CAD models, graphical multipass welding cross section handling, simulation, and welding camera integration.
The machine vision system uses FlexGui 4.0 as the user interface, with fully reprogrammable workpieces, filters, parameters, and playback. The vision system is robot and camera independent all through the ROS platform.
Trygve Thomessen, managing director of PPM Robotics, said: 'As a provider of the newest technology for industrial robotics, PPM would never survive by developing our solutions from scratch. Imagine, how to develop a cutting-edge 3D vision solution during three weeks, without ROS?'
PPM Robotics' ROSweld system
Vision system manufacturers are also recognising the growing demand for vision systems with an ROS interface. Today, many of the leading industrial vision equipment providers offer ROS interfaces for their products, such as Sick, Ensenso, Photoneo, and Zivid. The ROS Industrial 3D camera survey (https://rosindustrial.org/3d-camera-survey) provides a good overview of supported 3D cameras. Furthermore, a growing number of robot OEMs, above all UR, Pilz, ABB, Hanwha and Yaskawa, and component providers such as Keba or Bosch, acknowledge ROS as a de-facto standard for implementing complex robot applications.
Alexander Barth at Keba commented in 2019: 'Previously, integrating a sensor involved a huge amount of work. Now it’s all done by the community.'
Even though ROS has huge advantages for the development of robot automation applications, it also has drawbacks. Using open source software in products is different from using software provided under commercial license. In terms of liability, other rules apply, which the user needs to be aware of when planning to distribute ROS-based applications. Although the large ROS developer community enables fast innovation it sometimes creates short-lived software packages or those without maintenance support. Down the line, this can lead to problems in a ROS-based automation application. If you need help avoiding these drawbacks, contact your local ROS Industrial initiative.
Overall, ROS is a great software framework for machine vision-based robot applications. Robot OEMs, vision equipment and robot component providers, as well as robot application developers, are now using ROS, and we see more and more ROS-based robot applications being deployed into manufacturing plants.
About the author
Christoph Hellmann Santos is group manager in the robots and assistive systems department of Fraunhofer IPA. His group researches software engineering and system integration, and has a focus on open source robotics. He is also programme manager of the ROS Industrial Consortium Europe: Christoph.Hellmann.Santos@ipa.fraunhofer.de
Write for us
Are you using ROS or open source platforms to build vision systems? Let us know your experiences: greg.blackman@europascience.com
