The majority of hyperspectral imaging cameras used in industry today are line scan devices based on push broom sensors. Snapshot technologies provide an alternative, capturing an entire hyperspectral data cube in one sensor readout without needing to combine images.
Now, an EU project has recently begun to build SWIR hyperspectral chips, while an Italian firm has introduced a hyperspectral camera that uses Fourier transform technology.
Snapshot hyperspectral sensors have been around for a decade, with the advantage of easy integration into industrial equipment and fewer motion artefacts compared to line scan devices.
A compromise often has to be made, however, between spatial and spectral resolution when designing hyperspectral snapshot cameras, something that German company, Cubert, found when building its early cameras. The firm’s first Firefleye Q285 model offered 50 x 50-pixel spatial resolution, and could capture 125 spectral channels between 450 and 950nm via a prism-based sensor. It expanded its portfolio with the Butterfleye x2 camera, which, using a filter-on-chip, could provide an image resolution of 512 x 272 pixels, but could only capture up to 25 spectral channels.
Recently, Cubert has introduced a hyperspectral camera equipped with fundamentally new snapshot technology that enables it to offer both high spatial and spectral resolution. Called Ultris 20, the camera offers 400 x 400-pixel native image resolution and captures 100 spectral channels between 450 and 850nm.
According to Cubert, it is the first hyperspectral camera based on light field technology, where both the intensity and direction of incident light rays are used to capture data. Equipped with a 20-megapixel CMOS sensor, a continuously variable bandpass filter from Delta Optical Thin Film, and a lenslet array, the camera is able to capture multiple images of a scene in a single shot – each image taken with an optical bandpass filter featuring a different centre wavelength.
The camera can acquire images at the equivalent of a push broom sensor running at 400 spectral lines per image cube. With a frame rate that can reach up to 6Hz, this equates to 2,400 lines acquired per second.
Hyperspectral video
The ability to capture up to six complete hyperspectral data cubes per second means hyperspectral videos can be recorded using this new camera, a feature that has lots of potential, although demand for video at the moment remains low, Dr András Jung, co-founder of Cubert, told Imaging and Machine Vision Europe.
Jung said that one application for hyperspectral video could be analysing the mixture of ingredients in food batch recipes. ‘When determining whether a mixture has reached the ideal composition, it could be done using spectral information,’ he explained. ‘If companies could save minutes or even seconds per batch through using hyperspectral imaging, over time this would result in increased efficiency in their production.’
Mosaic filters are being integrated into InGaAs detectors to facilitate SWIR imaging as part of the Multiple project
One bottleneck to working with hyperspectral video is there isn’t any software suited to handling large amounts of spectral video data. This could change in the coming years, however, should demand for hyperspectral video rise. Jung expects the first firms to react will be those experienced in creating software for hyperspectral imaging, in addition to startups that emerge to address this demand.
While the Ultris 20 provides excellent spatial and spectral resolution, Jung remarked the micro-lens array of its light field technology does take up a considerable amount of space on the sensor, leaving fewer pixels for spatial resolution. He said larger detector arrays need to be made before the spectral and spatial resolution of this camera can be increased further.
Shrinking cameras
An alternative approach to developing hyperspectral snapshot cameras is currently being taken in the Horizon 2020 project Multiple, which began at the end of last year and is being co-ordinated by the Aimen Technology Centre in O Porriño, Spain. The three-year project was established with the aim of developing scalable monitoring solutions for different industrial processes using a range of photonic technologies, which, in addition to hyperspectral imaging, will include organic electronic sensors and laser-based spectroscopy.
The hyperspectral cameras under development will cover the SWIR range – from 900nm to 1.7µm – and will be based on InGaAs detectors. Currently in the prototype stage, these detectors are equipped with linear filters. In the future they will be based on mosaic filters developed by Belgian research institute Imec.
The hyperspectral technology being developed in the EU Multiple project will be used to measure the temperature of steel slabs on production lines. Credit: Russula
According to Dr Roi Mendez-Rial, a research engineer at the Aimen Technology Centre and co-ordinator of the project, the mosaic filters will either be positioned directly on top of the InGaAs sensor using a hybrid integration approach, or integrated monolithically with the sensor.
In order to make the new hyperspectral technology fast and compact, each camera will be developed using a mosaic filter that includes only a specific set of spectral bands suited to the application it has been designed for – rather than covering a complete spectral range.
One such intended application will be in steel factories, where the cameras will be used to measure the temperature of the walls of furnaces independent of emissivity, as well as the temperature of moving steel slabs on production lines. Woodwork and food production are also key areas that the project partners are targeting, and so end users from all three sectors are involved in the project. Food firm Yiotis, for example, will use the technology to monitor its chocolate production line. The steelwork and woodwork end-users are Russula and SCM Group respectively.
Although not part of the project, Mendez-Rial noted that another potential application could be inspecting the melt-pool of laser-based additive manufacturing, to obtain thermal and geometric features in order to facilitate closed-loop control.
The technology under development in the Multiple project also includes a compact dual-aperture camera that features both CMOS and InGaAs detectors with mosaic filtering. The device will be capable of imaging in both the visible (0.4 to 0.9µm) and SWIR (0.9 to 1.7µm) range, and according to Mendez-Rial, will be more compact and affordable than push broom cameras. The CMOS and InGaAs detectors are being developed by Imec, and the camera platform provided by Swiss imaging firm Photonfocus.
Low-light imaging
Last year, Italian firm Nireos introduced what it says is the first hyperspectral camera to use Fourier transform technology.
The Hera camera can only capture stationary objects, which differs from line scan equipment. Nireos is therefore targeting uses such as monitoring plants in greenhouses or imaging works of art, rather than conveyor belt or drone-based applications, according to Lukas Ceizaris, its business and product development manager. Another potential application is biophotonics, in which the camera could be combined with a microscope for studying biological samples exhibiting transmittance, reflectance or fluorescence.
The camera suits these applications because of its high sensitivity, which Fabrizio Preda, CEO of Nireos, said is a result of the Fourier transform approach.
Hyperspectral image of stained glass. Spectra at points 1 and 2 show that the two blue tiles are made of different materials. Credit: Nireos
‘Compared to the dispersive- or grating-based cameras on the market, our Hera camera can achieve a much higher throughput of light,’ he said. ‘The camera is, therefore, much more suited to imaging in low-light environments where other types of hyperspectral camera would usually require very powerful lights, which can even damage the sample.’
He explained that the Hera camera is so sensitive that in biophotonics it is possible to acquire a hyperspectral image of the fluorescence signal emitted by a sample, which can be several orders of magnitude weaker than excitation lights typically used to induce fluorescence.
Rather than capturing an image line by line, Hera acquires multiple images of a scene using different parameters via an interferometer at different points inside the camera. Thanks to the Fourier transform approach, the spectrum is a continuous curve – it contains information on all wavelengths in the range measured, rather than a selection of discrete wavelengths. ‘As a result, you are gathering a lot of information which standard push broom technologies, or cameras based on filter wheels, are unable to capture,’ said Preda.
The development of the Hera camera came about after Nireos had developed and sold Fourier-transform-based interferometers. ‘We noticed that they didn’t introduce spatial distortion, which led to us developing imaging equipment using the same technology,’ said Ceizaris.
Nireos has been receiving interest in Hera over the past few months, particularly from the scientific market.
Fantastic filters
Early last year, US firm Salvo Technologies acquired Ocean Insight’s Pixelteq optical filter line. As a result, it can now make its own linear variable filters, and is able to integrate them directly with off-the-shelf image sensors.
‘These filters cover a different part of the spectrum as you move across it, with each row of pixels dedicated to a particular wavelength,’ explained Hugh Garvey, director of imaging at Salvo Technologies. ‘Our sensors can therefore be operated in a push broom mode, where either the camera or the subject being imaged is moving – so each part of the subject can be imaged by each row of pixels – for example in airborne applications.’
He added that these filters, which are complex to make and have only become available over the past few years, provide a very compact and cost-effective method for performing hyperspectral imaging.
As customers understand more about these filters, however, they are demanding more spectral coverage in shorter filters, according to Garvey. There is therefore a push to make these filters perform to a higher standard.
‘The resulting challenge, from a filter manufacturer’s point of view, is to try and get from one colour to another in as short a distance as possible – the slower the gradient is, the larger the image sensor required,’ he said. ‘We are improving our filter manufacturing technology so we can make a high-performance filter in as small a piece of glass as possible, which can then go on a smaller image sensor. This will lead to smaller hyperspectral imaging cameras.’
