When is red, red or crimson might seem to be a simple question but being able to automatically determine what colour is actually being printed on a label or sprayed onto a product’s surface continues to be a challenge as quality and productivity improves.
When it comes to colour imaging, it is knowing that the colour the user is looking at is correct and what that colour result means for the responsible manager. ‘The two main activities are how to perform calibration to measure colour and how to use the colour to make a decision,’ says Arnaud Lina, manager for processing and analysis tools at Matrox Imaging.
Calibration can be divided into two types, absolute calibration and relative. Absolute calibration is where the target is able to return the measurement that can be correlated with a human decision. It is providing a measurement that is proportional to the way a typical human observer sees colour. ‘This absolute calibration provides complex and rich information about the temperature of the colour, what’s illuminating the object and about the camera sensor filters, so, for example, how the filters filter out the light at a given wavelength,’ says Lina. ‘They [calibration protocols] have to be really accurate and understand the material’s physical properties.’
Vision engineers will start with many assumptions about the material they are imaging. While the CCD’s properties and illumination will be known to gain a true measure of the colour quality, a camera cannot really compete with a spectrometer. Lina says that colour calibration is needed, for example, in measuring the colour variation on printed packaging and making sure it is within set tolerances. He draws attention to the CIEDE 2000 standard, which is a measure of the minimum distance between two colours that can be distinguished by humans. ‘If this is what the vision system has to achieve then you have to do absolute calibration,’ Lina says.
The alternative is relative calibration. The measurement system can be taught what is required by showing it reference colours of what is wanted and what isn’t wanted. The system will then ‘compare the target image related to those other colours’, explains Lina.
But even once systems are calibrated, over time colour temperature can change and users will have to recalibrate, checking between absolute colour values or reference colours and target colours.
Lina also warns against complacency when using the same camera models. ‘If you deploy your system with cameras, with the same model, they can have subtle differences with sensor filters so the values of a given scene, with the same illumination, may vary.’ Calibration might have to extend to individual cameras as well as their position, and there also must be compensation for colour temperature modifications brought about by differing illumination levels. ‘The change of the colour temperature is called colour drift. This is one aspect of the developments [taking place] to provide customers with [software] tools to perform the calibration,’ says Lina. As well as calibration, users can be provided with software tools for colour matching.
Fredrik Nilsson is product manager for 3D cameras at Sick, based in Waldkirch, Germany. His company is finding that customers want more and more features in their imaging systems. ‘We have had 3D cameras on the market for more than 15 years. Three years ago we added the colour feature, and thus combined 2D colour imaging and 3D image sensing on the same sensor chip. We have realised it is a powerful tool to combine everything on the same sensor,’ says Nilsson. The camera can measure colour in red, green and blue; it can measure greyscale, which can be both with and without infrared. With these sensors, Nilsson’s products have six different channels of spectrum information plus range and laser scatter measurements. ‘We think this will be used more in the future, combining more imaging capability in one camera,’ says Nilsson.
Sick has found that combining more and more into the same sensor is a clear trend with customers. When the industry is building more compact systems, customers see a benefit in having everything in one device. However, Nilsson can see some drawbacks: ‘You could say there are some trade-offs. The ways you can arrange and separate the light sources needed will be a bit limited when you have all imaging collected into one device.’ What Nilsson is referring to is being able to have sensors positioned at clearly separated locations, which could be an advantage to minimise interference between light sources. The reason customers are still happy with this is the compact form factor of the system, combined with the economy and the flexibility to select the most appropriate measurements for the inspection task – this is viewed as being of real benefit.
Colour inspection can add important data as to the quality of the product. Credit: Sick
It also makes sense when the cameras and their sensor systems have to be integrated within existing production lines and machines. ‘You need to find a place for them, the new cameras,’ says Nilsson. ‘The trend today is to increase the production flow by, for instance, increasing speed or belt width. So you need cameras that can measure at higher rate and at higher resolution to cover a wider field of view with maintained accuracy,’ he says.
One new area that Nilsson’s company has looked into is examining the relationship between the measured colour and the height position of the object. ‘One thing we have started to look into is when we combine the height measurement with the colour, we can do height dependent shading correction and compensate for colour shifting that may occur due to height variations,’ he explains. The closer to the light source the brighter the image will be, but colours may also shift because they are caught at different angles to the camera. ‘By knowing about the height position of the object, I can also compensate for both the brightness and the colour shift to get a normalised colour image,’ says Nilsson.
Sensors are all important and Teledyne Dalsa has on offer its bilinear and trilinear sensors. Bruno Menard, Teledyne’s image processing group leader, comments: ‘Mostly, every Teledyne Dalsa camera has a colour version that comes in categories. In the past, we had prism-based colour cameras that were expensive but provided higher resolution because of the precise alignment of the RGB colours. Right now we give our customers new tech bilinear and trilinear sensors.’
According to Menard, the trilinear sensor is extremely precise in terms of colour alignment, although not quite as accurate as a prism. However, as a compromise between price and accuracy the trilinear technology, in Menard’s view, is the best option. Bilinear differs to trilinear in that it uses one full line of green and its second line is alternating red and blue. This provides less accuracy than trilinear, but it allows greater bandwidth per line instead of the three lines for trilinear. ‘The one thing we tried to focus on is integration of colour calibration inside the cameras, such that you get the highest fidelity of colour with respect to the real scene. Instead of using complex software to grab a chart and calibrate with a spectrometer, you can do it from a simple setting in the camera,’ explains Menard. To achieve this, factory-calibrated coefficients are determind using different lighting systems (LED, fluorescent, tungsten, etc) in order to generate different colour temperatures.
Martin Grzymek is Teledyne Dalsa’s director of sales in Europe and is based in Munich. ‘The benefit of colour imaging is seeing defects you may not see with monochrome.’ The problem with monochrome is the lack of distinctions between colours. Grzymek adds: ‘The challenge lies in maintaining colour consistency and the amount of data you have to process to monitor that. This means the speed of the colour camera gets more challenging as more resolution is required for specific action tasks, not to mention you have to keep noise levels low.’
‘There are many different objects you can inspect here. Applications we are dealing with include flat panel displays, PCB, electronics, and wood inspection, as well as food inspection,’ says Grzymek. ‘There is also high-speed document scanning and printing inspection. Depending on the individual requirements, bilinear or trilinear [sensors] have already been applied to all of these applications.’
Alignment, illumination, contrast, colour variation; the factors that have to be accommodated are many and the rate at which all can be detected, measured and acted upon is only increasing. Rather like perfection, the quest for the best possible colour image is never ending.