Despite having been a research tool for a long time, open source software is now translating into industry, including in systems using machine vision.
So says Christoph Hellmann Santos, research group leader of software engineering and system integration at Fraunhofer IPA. He is also programme manager of the ROS-Industrial Consortium Europe, an open-source robotics software project that began as a collaborative endeavour between Yaskawa Motoman Robotics, Southwest Research Institute (SwRI) and Willow Garage to support the use of ROS for manufacturing automation. The software repository, hosted on GitHub, was founded by SwRI’s Shaun Edwards in 2012. Since then, robot perception through vision capabilities – usually 3D imaging – has become a key part of ROS.
The main advantage and defining characteristic of open source software is that users have full control over the software stack.
‘It enables you to share your source code, which, while being very important for research, seems to be against industrial companies’ interests,’ said Hellmann Santos. ‘However, despite this, we are seeing a lot of companies using open source software. This is particularly the case in the IT world, where much of today’s IT infrastructure – with the exception of personal computers – is now running on open source software.’
Open source is particularly suited to research, as it can be very limiting to use proprietary software packages: without having access to the full source code, researchers aren’t able to add changes when looking to implement new features.
‘With open source software, you can manipulate the software stack as much as you want, which avoids having to do workarounds,’ said Hellmann Santos. ‘Not only is this of interest to research, but also to companies, as they also have circumstances where they’d like to have access to the source code.’
This is especially the case when a company is looking to develop specialised vision software, he explained, as it can be beneficial to have the freedoms granted by open source when making modifications, in addition to being able to benefit from the latest developments made by others in the open source community.
It is particularly in the development of imaging solutions that Hellmann Santos sees the most benefit for open source software in machine vision, accessed through libraries such as OpenCV or Point Cloud Library.
‘If they are just looking to use machine vision technology for an existing application, such as bin picking, then they’d most likely want to use specialised proprietary software, which itself might already use open source libraries internally,’ he explained. ‘However, for creating the control software for a new vision or automation solution, open source could prove advantageous, as it enables any bugs to be addressed in the software directly, without having to approach the company that made the software.’
Because of this, Hellmann Santos believes open source software is unlikely to appeal to vision integrators – who tend to work with established system solutions – but rather those developing automation software or vision software for new applications. ‘These companies would use open source software in their software solution and provide easy-to-use features such as a user interface, which they could then sell to vision integrators,’ he said.
Examples of applications that can be addressed using new vision or automation solutions, developed using open source, according to Hellmann Santos, include: defect analysis, surface detection and inspection, bin picking and planning a path across a surface for applications such as sanding and painting.
Open, but not free
For those considering taking on open source software as part of their business, an important point to note is that it is not free. While it is true that the software itself doesn’t carry a price tag, its use does incur costs. First and foremost, in-house programming competence is required to understand and manipulate the source code.
‘If you use proprietary software, you pay a company to support the software and solve any problems and bugs that may occur, and you have a high degree of dependency,’ Hellmann Santos said. ‘With open source software you don’t have the same type of support and dependency. What you have is an open source community that can help, but this support is neither guaranteed nor immediately to hand.’
It’s therefore advisable to have an expert in-house, who can address any issues that occur, or failing that at least be able to describe them to the open source community.
‘If they are issues that are also being faced by other members of the community, then they are typically solved very quickly,’ Hellmann Santos continued. ‘If it’s a problem specific to you and your company, then issues may take longer to resolve. In these circumstances there are companies out there who do offer support to open source users, but this support is a paid-for service.’
Sharing innovations
As well as the flexibility that can be gained through adopting open source software, another big advantage is that any updates to it can be accessed instantly. ‘The community you are part of is often filled with researchers working on new developments and the latest technologies, all of which you have access to as soon as they are released,’ said Hellmann Santos.
In addition to gaining access to developments made by researchers, users of open source software can also profit from developments made by other companies. ‘From my experience, there are a lot of companies that try to develop everything on their own,’ he added.
‘They write all of their vision stacks on their own and do not profit from pre-existing developments, as they think everything needs to belong to them. This is where you can profit from open source. You can use things that other people have already developed to boost your development, while sharing your own knowledge by contributing back.’
This concept might be a shock to the industrial world, as it is a completely different approach compared to how companies have operated in the past: the entity that makes a development is the one that owns it and the only one that can sell it. It is therefore important to keep in mind that, when choosing open source, a company must decide which parts of its source code can be exposed to the open source community, and which parts will be kept as intellectual property.
Another important factor that could influence if open source is adopted by the vision industry, said Hellmann Santos, is legality: ‘We’ve seen that numerous companies are not fully aware of how open source licenses work, as their legal teams have never come into contact with them. This is changing, however, as more and more companies use open source.’
With software quality often being criticised, the transparency of open source software will also play an important role in its adoption. ‘If you have an open source package, you can see how it’s written and the kinds of tests that have been performed on it, which reveals the good and bad parts,’ Hellmann Santos said. ‘With closed proprietary software packages, you don’t see that and need to rely on the manufacturer’s word.’
Whether or not open source software can be regarded as ‘ready’ for industrial machine vision applications – compared to established proprietary software – will come down to the application. ‘What you always have to do as a software developer or integrator is test your solution. If you write your software well using open source, then test it well, it can easily be as reliable as proprietary software,’ he said.
