If spectroscopy can be thought of as ‘chemical vision’, then spectral imaging takes this idea one step closer to conventional digital vision by projecting spectral profiles onto an image sensor. The technique is used in aerial surveillance and satellite imaging, whereby an area of land is scanned from above over a range of wavelengths to build up a data cube of discrete single wavelength images. Individual spectra can also be generated from any spatial point on the image.
This sort of hyperspectral scanning over a continuous wavelength range has many other uses, although mainly in a research environment. The Imaging Concepts Group at Heriot Watt University, for instance, is collaborating with the Cheltenham Eye Clinic to investigate using hyperspectral imaging as a non-invasive tool for diagnosis of retinal eye diseases like glaucoma and age-related macular degeneration. A hyperspectral image of the retina can map oxygenation and show reduced areas of oxygen consumption and any potential damage this might have caused.
The Belgian research institute Imec has developed hyperspectral sensor technology, suited to industrial imaging rather than as a research tool. Andy Lambrechts, integrated imaging team leader at Imec, says applications are diverse. ‘We’ve tried to develop generic hyperspectral sensors that are useful for many applications,’ he says, adding that one field where the technology is in high demand is in food sorting and food safety. The sensor’s spectral range, covering 600 to 1,000nm, would be advantageous in food inspection. ‘Food production companies have experimented with hyperspectral sensing for a while, but the cost point and the bulkiness and the slowness of the cameras was not meeting their requirements,’ Lambrechts comments. ‘Our technology can change that.’
Using its semiconductor manufacturing tools and integration capabilities, Imec is able to integrate hyperspectral filters directly onto a CMOS imager. This wafer-level approach to fabricating a hyperspectral sensor means the camera can be made more compact. ‘When the filter structure is layered on top of a commercial sensor, you take a camera that was built for the original sensor and replace it with the hyperspectral sensor to create a hyperspectral camera in the same form factor as the original. The software needs to be updated to make sense of the data, but the camera form factor remains the same,’ explains Lambrechts.
Imec has a line scan version of its hyperspectral sensor, based on a sensor from Cmosis, as well as a video-enabled area scan version where the object doesn’t need to be scanned in front of the camera. In the line scan model, 100 spectral filters are integrated to cover full columns of the sensor. The area scan version uses a Cmosis sensor subdivided into 32 segments in a checkerboard pattern of 4 x 8 with small square filters. A lens array then duplicates the scene onto the 32 zones of the sensor. The sensor therefore acquires an image in 32 different wavelengths in parallel in every frame without having to scan the object. This is very useful for inspecting objects that are moving randomly, or inspection in the field where it’s impractical to scan.
As well as food inspection, another possible application Lambrechts notes is integration into UAVs, because of the small size of the solution and UAVs typically being restricted on payload. He adds that the sensor technology is specifically designed to be generic to make it applicable to numerous areas.
‘There are multi-band video solutions available, but these use traditional components like gratings or prisms to capture different bands on a sensor,’ Lambrechts says. ‘These methods lead to expensive and large cameras, expensive because of the high-end optical components, but also because of the complexity in integrating them and aligning the whole system in a stable and robust form factor.’ Imec integrates the most sensitive components, mainly the spectral splitting parts, onto the sensor directly using its existing high-precision equipment from semiconductor manufacturing. ‘Our camera is tiny compared to other hyperspectral cameras and the sensor is built using mass-manufacturing processes and equipment,’ he adds.
Imec’s hyperspectral system uses an optical duplicator component around 1cm thick fixed between the objective and the camera to duplicate the image equally on all the sub-tiles of the sensor. Lambrechts comments that most other hyperspectral approaches require complex fore-optics.
Imec offers a full evaluation system consisting of a camera with the hyperspectral sensor, the software, a translation stage for the line scan version, and illumination. Imec can also integrate its hyperspectral filters on specific sensors to create custom solutions. The company is also planning to extend the sensor’s spectral range in the next version to cover from 400 to 1,000nm.
Selecting the right wavelengths
Because a hyperspectral sensor scans and gathers data across an entire band of wavelengths, the resulting data cube can contain too much information for some applications. It’s good for research, says Marco Snikkers, director of sales and marketing at Pixelteq, formerly Ocean Thin Films, but it often contains too much data for OEMs. Pixelteq provides OEM spectral imaging solutions, including its PixelCam and SpectroCam, both multispectral imagers, which Snikkers says, ‘provides just enough information at a price where OEMs can use the technology at higher volumes’.
Still classed spectral imaging, multispectral cameras have the advantage of improved contrast by imaging at different wavelengths, but only have a set number of channels and so produce a lower volume of data. In the same way, colour cameras will have an RGB pattern layered over the pixels, a multispectral camera will extend this capability by adding channels at other specific wavelengths.
‘You need to have enough contrast between the region of interest or the characteristic the system is looking for and the background,’ explains Snikkers. ‘For very specific applications, a multispectral imager will improve the contrast significantly by using small band-pass filters to identify spectral changes and enhance the features in the image against the background.’
Pixelteq uses lithographic processes to deposit micro-patterned dichroic coatings at a pixel level onto image sensors. Its PixelCam is based on commercially available image sensors, either visible, UV enhanced, or SWIR sensors, with individual pixel micro-filters coated according to spectral bands. The camera provides a real-time image similar to an RGB camera, but with specific wavelength bands integrated on the sensor chip.
Pixelteq’s SpectroCam is a monochrome camera with an eight-channel filter wheel, allowing customers to specify the wavelength with different filters.
Steve Smith, a product manager at Pixelteq, lists three growing areas for multispectral imaging, namely forensics, art investigation, and the biomedical field. ‘Most applications are able to narrow down the number of wavebands to make the analysis, which lends itself to multispectral imaging where you can use a CCD and specific 10nm band-passes tailored to the application,’ he says.
Biomedical analysis, and particularly that based on fluorescence, is making use of spectral imaging to discern between the light emitted from different fluorescent proteins in studies using a number of these labels. ‘A lot of fluorophores overlap in terms of the wavelengths of light they emit, so you generally need narrow spectral bands for imaging,’ explains Smith.
Spectral imaging can also reduce autofluorescence in samples, where the tissue is lit at almost the same wavelength as the sample emission, creating an unfocused image of the fluorophore. Alex Fong, senior VP, life sciences and instrumentation at Gooch and Housego, explains that in an immunohistochemistry assay in cancer diagnosis, for instance, often the boundaries between different stains, which delineate healthy tissue from malignant tissue, aren’t clear to the human eye using standard brightfield or transmission microscopy. Spectral imaging can help clarify those boundaries and could potentially be very useful in medical diagnosis and cancer research, according to Fong.
Photonics firm Gooch and Housego has developed its HSi-440C hyperspectral imaging system, which can perform unmixing or classification of the image in near real time. ‘Most hyperspectral systems collect the data first and then apply unmixing algorithms after the event. It usually requires several steps to do that,’ explains Fong.
‘Essentially the aim is to analyse the image and classify all the pixels according to spectral profile. It’s a computationally intensive process and normally it’s carried out sequentially, with a spectral image of a scene captured and then processed.’
The HSi-440C is able to process the data as it’s being acquired, due to the spectral imager’s ability to switch between wavelengths incredibly quickly. Switching to different wavelengths occurs at 100μs, around 1,000 times faster than competing technology, claims Fong. Combining that with a high-performance graphics processing card and advanced algorithms, means the system can capture and process the spectra in near real time – the frame rate is around 7fps and each band-pass is being corrected at 27.6fps.
‘Being able to separate out the various wavelengths in real time is useful to a biologist or a life science researcher in that they can set up the parameters to unmix one portion of a slide and then scan around the rest of the slide for whatever morphological phenomenon is of interest,’ explains Fong.
The HSi-440C hyperspectral system is based on an acousto-optical tunable filter (AOTF), using a nonlinear optical device based on a tellurium dioxide crystal. The wavelength changes are actuated by sending sound waves through the crystal from a transducer plated onto the side of the crystal. A radio frequency driver is used to stimulate the transducer and create the sonic waves. The amplitude and frequency of the sonic waves passing through the crystal determines the band-pass and the centring of the measurement wavelengths. With the HSi-440C, once the major spectral components have been identified the number of band-passes can be scaled down to speed up unmixing and reduce data compression and storage needs. A traditional push-broom hyperspectral sensor, on the other hand, collects all the band-passes whether they are needed or not.
Imec’s hyperspectral solution makes spectral imaging technology suitable for industrial inspection, while Pixelteq targets OEM customers with its multispectral cameras. A number of machine vision software packages, including Halcon from MVTec, also have the functionality to process images with multiple channels of data and extract the required information. Halcon is supplied in the UK by Multipix Imaging.
Already, a lot of industrial inspection applications make use of an infrared channel along with the three RGB channels to gain extra contrast in the image, and as the complexity and price of spectral imaging systems come down, more applications will open up.
