HC-Vision's early prototype rig, a precursor of its single-sensor hyperspectral imaging camera
In recent years, the amount of commercial and industrial applications of hyperspectral imaging has continued to increase. So, what are the main current and potential industrial and commercial applications? What are the key benefits and challenges of using the technology? And what innovations and trends can we expect over the next few years?
Farm to fork
For several years, hyperspectral imaging has been used for aerial photography in agriculture and forestry, for example to detect forest damage and record forest dieback. The European Union uses it for regular aerial inspections of farmland, making it possible to determine whether grants for growing a certain crop are justified.
Farmers can use hyperspectral imaging to determine how much fertiliser to apply to crops. The technology can also indicate when crops are ripe for harvesting – constituents of fruits, such as sugars and acids, emit certain colour spectra, which hyperspectral imaging can detect through the peel. This opens up the possibility for automated harvesting with devices like apple-picking robots, or sorting individual wheat varieties into different quality grades. The technology also supports automated potato harvests, so that tubers can be distinguished from stones of similar appearance.
Elsewhere, hyperspectral imaging is increasingly used for quality control in the food industry, where it can be used to detect otherwise invisible layers of mould on bread or other foods, or establish the quality of fresh meat by determining fat content or levels of bacterial contamination. ‘In contrast to conventional cameras, hyperspectral imaging detects dents under the peel of an apple that are not visible to the naked eye,’ explained Dr Wolfgang Eckstein, co-founder of MVTec Software. ‘This enables poor-quality fruits to be automatically and reliably sorted out, before they are put out for sale.’
Mark Williamson, director of corporate market development at Stemmer Imaging, agreed that food analysis is a major market, and pointed to interesting applications like the identification of insects and animal remains in pulses and grains with ‘very positive near-100 per cent detection rates’.
Other possible application areas include mining, where Eckstein said the technology is used to separate ore from rocks, and recycling, where it is employed for plastic sorting and wood recycling, as well as to identify a wide range of components in rubbish, helping to optimise the separation of reusable materials from waste. The pharmaceutical industry also benefits from hyperspectral imaging techniques, where the ingredients of medicines can be determined to make sure the right drugs are placed in the right packaging. It can also identify counterfeit pharmaceuticals.
Detecting complex patterns
Eckstein noted hyperspectral imaging is of key interest for machine vision applications, with the main benefit being that it ‘reveals things that remain hidden to conventional cameras and even the human eye’.
‘Hyperspectral cameras can identify certain objects and scenarios much more effectively by reliably detecting material properties. For example, Halcon, MVTec’s machine vision software, contains a large number of functions that can be used to identify objects through the gradation of different grey-level values,’ Eckstein said.
‘The slight differences in colour tone indicate certain product properties and detect faults or damage. As a result, such technology is also an excellent choice for inspection tasks and thus for quality assurance,’ he added.
Williamson agreed that hyperspectral imaging can see things that normal imaging can’t, meaning new applications are now possible that were not 10 years ago. Until very recently, he said that an industrial hyperspectral imaging system cost in excess of £100,000 because of the custom nature of the solution, reserving it for applications that could carry such a cost. However, he believes that recent developments in classification technology, combined with advances in traditional machine vision tools, mean that historically complex areas requiring scientific expertise can now be addressed by industry-focused integrators.
‘We have also seen the complexity of spectrographs coupled to cameras being replaced by pre-calibrated hyperspectral cameras such as the Specim FX10 and FX17. This makes some applications even easier to deploy. Prices for a complete system are now in the £50,000 area and I expect prices to reduce further,’ he said.
Hyperspectral cameras can distinguish between different plastics in recycling plants. Credit: Perception Park
Manfred Pail, CFO at Austrian firm Perception Park, pointed to the fact that hyperspectral imaging essentially entails ‘the spatially resolved industrialisation of spectroscopy’ as another key advantage. Perception Park, among other things, delivers an enabling chemical colour imaging technology that allows users to develop their own hyperspectral applications.
‘So spectroscopic and chemometric features – chemical properties – can be processed in real-time. Through statistical and chemometric methods, starting with just calculating derivatives along the spectral information, up to performing principal component analysis (PCA) or partial least squares (PLS), a deep understanding of the condition, state and even the chemical composition of an object can be evaluated,’ Pail said.
Towards commoditisation?
According to Boaz Arad, co-founder of Israeli hyperspectral imaging start-up HC-Vision, a common theme relating to most of the existing applications is the fact that they tend to be implemented by very large, well-funded organisations, such as multinational companies or government entities. In recent years, he has also observed a push towards the commoditisation of the technology, with hyperspectral acquisition hardware becoming smaller, cheaper and faster.
‘Crop monitoring solutions that were once exclusive to large agricultural concerns are now being made available to individual growers by start-ups such as Gamaya. While not quite a hyperspectral imaging system, Consumer Physics’ pocket molecular sensor SCIO made quite a splash by providing consumers with an affordable and user friendly point spectrometer,’ Arad remarked.
An example of the low-light performance boost the HC-Vision camera design provides over conventional RGB cameras. Credit: HC-Vision
‘At HC-Vision we are pushing the envelope even further, by converting existing RGB sensors into VNIR hyperspectral imagers. Since these sensors are already ubiquitous in consumer electronics such as mobile phones, we believe this innovation may be the key to bringing hyperspectral imaging to the masses,’ he added.
In Arad’s view, the wealth of data provided by hyperspectral imaging now affords industry a ‘unique opportunity to detect complex patterns’ – with recent advances in machine learning ‘further leveraging this advantage, allowing the rapid development of detection algorithms and analytical processes’.
Remaining challenges
Despite the undoubted benefits, Eckstein stressed that a number of key challenges remain before mass adoption can be achieved. He pointed out that the calibration of small cameras and microscopes is still an extremely complex process, with current calibration blocks suitable for lab applications only.
‘Light conditions are an enormous challenge for developers, since a suitable, invariable light spectrum is always needed. However, this is unrealistic under real conditions. After all, the light spectrum varies greatly from one application to another, and must therefore, be continuously recalibrated. This requires an enormous amount of work,’ Eckstein remarked.
‘Experts believe that it will take at least another five years before hyperspectral imaging and corresponding applications will have progressed enough for use on the mass market,’ he added.
Meanwhile, Arad warned that the cost and complexity of current hyperspectral systems are ‘still major hurdles in their widespread adoption’. Although the cost of single acquisition systems may be affordable even for a small business, he argued that the added cost ‘becomes unsustainable when multiple imaging systems must be deployed’. In his view, another problem is that the industry has not yet really developed set solutions for hyperspectral data processing, with most applications still requiring a time consuming, and possibly expensive, development phase.
Fortunately, he said that quite a lot of work is being done to overcome this limitation, with companies like Perception Park already reducing the development time of such solutions, by making hyperspectral data more easily readable by non-experts.
Hyperspectral classification of land use from an aerial photograph. Credit: MVTec Software
‘As for the issues of cost and complexity, I believe we’ll see two distinct trends emerge,’ Arad said. ‘The cost of dedicated hyperspectral imaging equipment will continue to fall, making it a viable option for small businesses and even hobbyists. At the same time, the spectral sensing capabilities of conventional image sensors will increase significantly.’
‘VNIR sensors are already quite common in cellular phones, often referred to as RGBW, though their spectral capabilities remain hidden behind the camera’s image processing hardware. With companies like HC-Vision tapping into these abilities, and perhaps even enhancing them through small changes in camera design, it’s likely that we’ll see hyperspectral sensing applications in the consumer market sooner, rather than later,’ Arad added.
Cost reduction
The amount of data passed from a hyperspectral camera tends to cause a high load for processing units, typically calling on devices like PCs to reduce the data by extracting relevant spectral information. However, in the future, Eckstein has predicted that this kind of reduction and pre-processing could be carried out by the camera directly – following an application-specific configuration – helping to ‘significantly reduce the cost and therefore broaden the range of applications’.
‘The other critical issue is the calibration of the camera and light setup. Currently, there is no standardised form to perform that. Having such a standard would ease the realisation of applications and better support quality assurance,’ Eckstein said.
Williamson agreed that increases in GPU processing capabilities mean that the ‘vast levels of processing needed in real-time applications are becoming more feasible’. Ultimately, he believes it will not be long before there are smart hyperspectral cameras with pre-defined classifiers getting closer to the application.
‘Hyperspectral imaging is in its infancy and first we need to see the quantity of applications being successfully solved. Once we see take-up using the recent developments, we will see costs reduce further, in turn making more applications viable,’ Williamson said.
Looking ahead, Pail envisages a ‘massive’ drop in the price of hyperspectral cameras, accompanied by dramatic miniaturisation and increased resolutions. He pointed out that, in 2008, it was nearly impossible to purchase a hyperspectral camera for less than €50,000 and the cameras were large and bulky, but that today, ‘cameras a quarter of the size are available for less than €20,000’.
‘So in the future we will also find hyperspectral cameras for a few hundred Euros. As a consequence, more and more applications will become feasible and hyperspectral technology will find its way into many industries and into the consumer market as well,’ Pail added.
Over the next few years, Arad thinks that the most exciting area of innovation for hyperspectral imaging is going to be the consumer market – and he pointed to huge demand for hyperspectral applications such as food analysis and safety, drug verification, forgery detection and even security and biometrics.
‘The current generation of hardware and software can’t quite meet the demand just yet, but we’re fast approaching the point where it will,’ Arad concluded. ‘Those who anticipate the expansion of hyperspectral sensing into consumer applications and position themselves accordingly will reap the benefits.’
