Colour imaging usually refers to visible light, but extending an image sensor’s range to incorporate shorter or longer wavelengths is a focus of much study. Researchers at RMIT University, in Melbourne, have developed a prototype photodetector that can see all shades of light, between ultraviolet and near infrared, where previously they were unable to sense more than one colour in one device. It is also around 1,000 times thinner than the smallest commercially available photodetector device.
Their challenge was that, generally in photodetection technologies, reducing a material’s thickness can come at the expense of performance. Study lead author Vaishnavi Krishnamurthi explained: ‘We managed to engineer a device that packs a powerful punch, despite being thinner than a nanometre, which is roughly a million times smaller than the width of a pinhead,’ she said.
Magic number
The versatility and usefulness of photodetectors is dependent on three factors: operating speed, sensitivity to lower levels of light and how much of the spectrum they can sense. Krishnamurthi said: ‘Typically, when engineers have tried improving a photodetector’s capabilities in one of those areas, at least one of the other capabilities has been diminished.’
The researchers scrapped the traditional stacked model, which relies on a stacked structure of three to four layers, and worked out how to use a nano thin layer, a single atom thick, on a chip. They did so without diminishing the photodetector’s speed, low-light sensitivity or visibility of the spectrum.
The prototype device can interpret light ranging from deep ultraviolet to near infrared wavelengths, making it sensitive to a broader spectrum than the human eye. It does this more than 10,000 times faster than the blink of an eye.
One of the team’s major challenges was ensuring electronic and optical properties didn’t deteriorate when the photodetector was shrunk. To combat this they used tin monosulphide, a low-cost and naturally abundant material.
Associate professor Sumeet Walia, the chief investigator, said: ‘The material allows the device to be extremely sensitive in low-light conditions, making it suitable for low-light photography across a wide spectrum.’
Need for speed
The development means that this type of technology could be used in new applications. Walia explained: ‘With further development, we could be looking at applications including more effective motion detection in security cameras at night and faster, more efficient data storage.’
Developments have also been taking place when it comes to colour imaging in the commercial arena, and these look set to prove beneficial, not just for medical, but a host of other applications.
JAI is a manufacturer of industrial-grade cameras for the machine vision market, among others, and has developed a range of multi-sensor area scan and line scan cameras that provide high-end colour reproduction. It recently launched 4k and 8k prism-based line scan cameras with 10GE and SFP interfaces.
Product manager Paritosh Prayagi said: ‘Trilinear sensors are structured with individual lines – that’s R,G and B when you’re talking about colour imaging. You have the sensor on the same physical plane, but the point that it looks on the object is very different: an object moving at an unknown speed, or something that is rolling or in freefall, for example.
‘Many fruits and vegetables roll on a belt, or are dropped from a specific height with rotation, which can cause problems when using trilinear cameras. This is because, if they are not stable, the trilinear camera cannot compensate for the movement in high-speed.’
This is an area where a prism-based camera comes into its own. The prism separates the light onto three sensors to get the RGB image. ‘Basically, there is a single optical access that is reflected from the object, and then we split the light inside the camera,’ Prayagi explained.
The right path
This method is beneficial because it means that there is no need for optical compensation, as there is only one path to the object, as opposed to trilinear’s three for RGB. ‘It actually results in much better colour information,’ said Prayagi, as there is no blur, and no need to compensate for the positioning mismatch from the object’s surface, so you get a better colour.’
Objects with anomalies, such as the wrong colour or size, can be identified for rejection. Credit: Sick
Another advantage of the prism approach, said Prayagi, that also results in better sensitivity, which influences the colour, is that there are three individual sensors in the prism approach. ‘This means that we can control the analogue gain, which is the gain on the sensor level. It’s basically the signal amplification and we can control the exposure time for individual sensors.’
Prayagi went on to explain that CMOS sensors are more sensitive on the green wavelengths, so a CMOS sensor projected onto a surface for white balancing will produce a very green-dominant object. ‘The red and the blue channels then need to be balanced, in order to see a white background,’ he said. ‘In trilinear cameras, you pull up the signal of the blue and the red channel to match with the green, but in the process of doing that, you also amplify the noise, which is not ideal.’
The prism approach, however, provides the opportunity to amplify the gain, but also change the exposure time per channel, thanks to the three individual sensors. This way the noise does not have to be amplified. ‘This results in a better colour reproduction,’ said Prayagi. ‘If you have good sensitivity of a vision system, it means that you can produce better colour.’
In terms of the evolving applications for colour for which the prism-based line scan camera allows, one area where it is proving particularly effective is web inspection. Prayagi said: ‘In this type of application, the web is moving at very high speeds and it can migrate, to an extent. There are many mechanisms to reduce the vibration of the webs. It’s a 200- to 300-year-old discussion, how to remove the vibration of moving objects while printing, but still there is a vibration, and this vibration can be visible in the image.’
Mind the gap
Trilinear technology is forced to compensate for the line gap caused by the vibration, between the R, G and B lines, which is achievable, said Prayagi, if the vibration is constant. ‘Of course, in reality it is not. So you start to see these colour fringes.’
Digital colour imaging can be used for disease diagnostics and monitoring progression of a disease. Credit: JAI
Another option is to use dual line technology, which is a bayer-pattern line scan. But, Prayagi added, there still needs to be compensation for one line, because, while there are two lines, there is a gap between those lines. ‘This also results in the colour fringes,’ he said, ‘which are less than with the trilinear, but still there.’
The prism approach, however, because of its single optical access, means that it is less relevant whether there is vibration or not, because there is the same effect in all channels. ‘There is no need for any compensation there,’ explained Prayagi.
Colour imaging, in the form of Bayer mosaic imaging, was developed in 1974, when US scientist Bryce Bayer invented the Bayer filter while working for Kodak. However, Prayagi said, the resolution at that time was very low. ‘The real resolution of blue, for example, is not exactly all the pixels on the sensor, it is probably only one third.’ This problem was more recently overcome using the prism approach.
Has colour imaging now reached its peak performance, or can it be developed further, opening up more new and exciting applications?
The latter thinks Prayagi. ‘It can get better, but mainly in terms of multi-spectral imaging. The advantage we have with the prism approach is that, if you want to have a really special colour reproduction, for example, with sorting fruit, you need to have a certain waveband that is much more narrow than a standard sensor waveband.’
A recent launch from the company has been designed for just this purpose. The Flex-Eye configurator tool can be used by customers as a multi-spectral solution.
‘You can define special wavebands,’ said Prayagi, ‘that cannot be used for a standard RGB camera.
‘The result is that you can have special responses in RGB, and also near-infrared imaging combined, which are customised to fit a specific application.’
Colour imaging via an app
Sensor solutions manufacturer Sick has also seen an increase in demand for colour imaging. It has developed a colour inspection and sorting app, as part of its SensorApps range for simple machine vision setup.
Among its applications, the app can be used to check that goods, assemblies or packs on a conveyor belt are the right size or colour. It can count objects with different sizes and colours, as well as validate the correct colour or colour gradations, for example of baked goods, or automotive components in an assembly. Objects with anomalies, such as the wrong colour or size, can be identified for rejection, and the integrity and completeness of secondary packaging can be confirmed.
It is designed to work with the company’s SIM4000, SIM1012 or SIM1004 cameras.
The solution was developed with ease-of-use in mind. The object colour can be matched by using the colour scale or by simply inputting the colour pigment. Object sizes are easily defined by minimum and maximum pixel values.
Multiple use cases
The potential applications for colour imaging could be anything from intelligent farming, where it can help to monitor plant health and growth, to inspecting lice on salmon skin for fish farmers, to medical and surgical guidance. Prayagi explained: ‘One use is for early detection of tumours, where the big challenge is to be as accurate as possible in surgery. So, the cut on the body needs to be as small as possible and this can only be done if it is accurate, so we know what the trouble area is.’
In this scenario, the patient is injected with indocyanine green (ICG) dye, which clings on to the tumours, allowing them to be optically identified using prism-based cameras.
The number and type of applications for colour imaging is growing, according to Prayagi, such as pharmaceuticals and cosmetics, for identifying defects in drugs or their packaging, as well as electronics, meat and poultry inspection, and inspecting currency and passports.