By 2020, according to EU directive 2008/98/EC, the quantity of household waste recycled in European countries should have been ‘increased to a minimum of overall 50 per cent’. Recycling is now big business, as plants are asked to deal with a wider variety of recyclable materials and to process larger quantities of waste. In all, 64 per cent of household waste in Germany is recycled, the highest in Europe, and this is possible, in part, thanks to advances being made in hyperspectral imaging.
Hyperspectral imaging can distinguish between different materials by analysing their spectral data – each material has a unique chemical fingerprint. Recycling centres use the technology to sort waste, and faster hyperspectral cameras that are easier to use are making this process more effective. These new and improved systems can be integrated easily into process chains and operated with only a small amount of training, leading to an increased adoption of hyperspectral technology across other industries. Both food and textiles are growing end users of the technology.
In recycling plants, hyperspectral cameras are usually positioned above a conveyor belt in order to sort the waste. A broadband artificial light source next to the camera is used to illuminate the contents of the belt. ‘In many cases, there will be multiple cameras in the whole recycling chain,’ said Murali Jayapala, a principal research and development engineer at Belgian institute Imec. ‘One can identify foreign, unwanted objects, another distinguishes between different types of material, and a third performs a final classification or re-distribution of the materials further down the chain.’
A compact high-speed camera with Imec's hyperspectral sensor
As the contents of the belt passes underneath the camera, the light, which is typically in the visible to near infrared (VNIR) wavelength range, illuminates the material and interacts with the matter. Depending on the chemical composition of the material, different wavelengths of the broadband light are absorbed – some more than others – while the rest are reflected back into the camera. The system determines the chemical composition of the material – and thus its identity – by analysing which wavelengths have been absorbed. ‘Often in these cases the data captured by the sensors is processed in a separate computer in the system,’ said Jayapala, ‘However, you also see that, in smart cameras, the decision-making is embedded inside the camera.’ By moving the data processing step inside the camera, manufacturers of hyperspectral systems have reduced the size of their footprint dramatically.
Smaller, simpler, faster
Reducing the size and cost of hyperspectral cameras is an important step that will lead to greater adoption in industry. Imec has developed a hyperspectral line scan sensor, in which the spectral unit has been integrated on top of a standard CMOS sensor at the same thickness as the rest of the circuitry on the semiconductor wafer. Most hyperspectral cameras use a combination of prism, collimator and grating components; however, Imec puts micro-filters on its sensor, removing the need for complex optics. This reduces the cost and improves the compactness and speed of the hyperspectral cameras built with the sensor.
‘Normal hyperspectral sensors usually work by capturing the information one line at a time or one point at a time, which limits the throughput of these conveyor systems,’ observed Jayapala. ‘Using Imec’s sensor, the throughput can be increased significantly because it has a much higher spatial resolution and can capture far more light and much more information at a faster rate.’
Imec’s hyperspectral image sensors offer 2,048 x 1,088 pixel resolution and operate in the 470-900nm (VNIR) or 600-1,000nm (NIR) ranges, offering 150+ or 100+ spectral bands respectively. The sensor consumes 600mW of power and can achieve a scan rate of up to 2,720 lines per second, depending on the limitations of the camera interface harbouring it.
Dutch camera maker, 3D-One, has integrated Imec’s sensors into a hyperspectral camera called Avior, which is ideal for recycling or sorting applications. Marco van Hout, business manager at 3D-One, commented: ‘We have developed an extra feature [on Avior] that allows users to select the data they wish the camera to obtain. It reduces the amount of data enormously, especially in hyperspectral imaging as users are confronted with large amounts of data anyway, and a big chunk of this is waste.’
During the operation of standard hyperspectral imaging systems, large amounts of spectral data has to be stored from across a broad range of wavelengths. By pre-defining a dataset of wavelengths for the camera to analyse selectively, the amount of data being recorded is cut down dramatically, which increases the acquisition speed of the camera, contributing to a higher throughput of the entire system.
Datasets can be predefined by knowing how certain objects reflect light. In a recycling centre, a set of paper, plastics and glass would be analysed at an earlier stage to determine their absorption patterns. Once the desired wavelengths are identified, the camera can then be pre-programmed to detect only those spectra. The increased throughput resulting from this selective analysis will enable recycling centres to deal with larger amounts of waste, allowing them to address the increased pressure from their governments trying to achieve EU targets.
Compact, lightweight hyperspectral imaging systems – such as those that include Imec’s line scan sensor technology – face fewer complications when being integrated into new systems, as they are easier to mount. Compact cameras are also more manoeuvrable, increasing their suitability for a wider variety of applications.
The robustness of the camera is also important, as those installed in recycling plants are subject to high vibrations and extremes in temperature. ‘We chose Camera Link as an interface to provide robust transmission for digital video,’ van Hout commented. Lower sensor operating temperatures correspond to a better signal-to-noise ratio; therefore, a robust camera with a thermo-mechanical design will produce higher quality data.
Potentially the most important change that will improve adoption of hyperspectral imaging in industry is to make the systems easier to use. Previously, experts have had to be brought in to set up imaging applications, leading to higher integration costs. Simplifying the use of hyperspectral equipment is exactly what Perception Park has achieved through its new interface, the Perception System – the company’s technology was shortlisted for the Vision Award at the Vision show in Stuttgart in November.
‘The Perception System is a configurable data processing solution that enables the industrial use of hyperspectral cameras, especially for high-speed tasks such as sorting in recycling and food processing,’ explained Manfred Pail, CFO and in charge of marketing and sales at Perception Park, the Austrian hyperspectral data processing specialist.
The new solution presents an alternative way of perceiving hyperspectral data through a technique known as chemical colour imaging. The complex information collected by a camera analysing a set of materials is converted to a stream of both monochrome and multi-coloured images, with each material’s chemical composition represented by a different colour.
According to Pail, the Perception System can be seen as an adapter that turns a hyperspectral camera into a chemical colour camera.
‘We present the spectroscopic features of a material as information that everyone can understand and interpret,’ he explained. ‘All the scientific methodology that is known in the lab has been encapsulated and provided through an interface so that everyone can use hyperspectral cameras.’
With the new interface, plastics, for example, could be identified by their type, based on the colour to which their spectral information translates. PET plastics could appear one colour, while PVC plastics another. ‘It’s not a colour classification, just a colour representation,’ commented Pail. ‘We are bringing science to the engineering level. You don’t need scientists any more to develop a hyperspectral application; every engineer can do it after one or two day’s training from Perception Park.’ In using the interface, companies can save development times by as much as 95 per cent.
Taking things further
The faster, smaller and now simpler hyperspectral imaging systems are being used in industries other than recycling; both food and textile factories now use the technology in everyday operation.
‘We have been involved in a project for the past three years called Sensors For Food,’ said Jayapala of Imec. ‘Here the whole idea was to evaluate whether hyperspectral could play a role in different aspects of food inspection.’
Through a set of feasibility studies, the project showed that hyperspectral data can be used for food quality control to check for aspects such as ripeness, bruising and impurities, as each will demonstrate a unique spectral signature.
The project was completed at the end of 2015, with a subsequent scheme called i-FAST building on the work. Jayapala explained: ‘The idea here [with i-FAST] is to translate many of these feasibility studies from Sensors For Food into the field. We received good feedback from many of the companies, so now we are taking these hyperspectral systems and going to their production centres to see how we can integrate the cameras into their systems.’
Food production is now integrating hyperspectral cameras to check crops like potatoes for flaws, such as bruising.
For mass inspection applications – such as those in factories processing large quantities of meat, fruit and vegetables – a standard conveyor-belt system is used. However, for instances where a more detailed set of inspection images needs to be obtained, a work table setup or a hand-held hyperspectral device could be used. As in the recycling industry, controlled illumination would give the light needed for spectral analysis on a case-by-case basis.
Hyperspectral cameras in food production take multiple images of each item at different orientations and angles, and applications such as this have led to further developments in the technology. Jayapala said: ‘Hyperspectral technology for food can be much more sophisticated in terms of image processing and data handling because it has to account for orientation and rotation.’
Imec is also currently involved in Resyntex, a Horizon 2020 project aimed at establishing a complete chain of textile waste collection, recycling and the redistribution of the chemical by-products. However, the adoption of hyperspectral imaging by the textile industry has so far been limited because of the lack of fast, compact and cost-effective hyperspectral cameras with adequate specifications. Imec aims to change this with its line scan sensor solution.
‘If you have a complex textile which has cotton mixed with polyamide at different parts, a point-based imaging approach would give you a single point that could be interpreted as cotton, a polyamide or a polyester,’ said Jayapala. Using the spatial information provided by a line scan sensor, a gradient can be observed that enables a much more accurate estimate of the composition of materials in the line.
As part of the Resyntex project, an Adimec camera – featuring the Imec sensor – was positioned above a conveyor belt. The system achieved a speed of 1,080 lines per second, enabling a conveyor speed of around 0.5m/s, enough to process one cloth item per second. The initial pilot system in the Resyntex project is expected to process up to 100kg of textiles per hour. The higher hyperspectral imaging throughput now available for plastics and paper recycling has therefore benefited textiles recycling. The technology is also now increasingly relevant because more textiles are being incinerated or buried in landfill.
The use of hyperspectral imaging is becoming more widespread in industry, and Pail believes that, once the technology has advanced further, hyperspectral systems could become modular, enabling them to be produced by adding spectrographs in front of standard cameras. ‘This could be something that comes in the next two or three years,’ he said.
The progression of hyperspectral imaging in industry will be the focus of the Conference on Hyperspectral Imaging in Industry, taking place in Graz, Austria from 7 to 8 June 2017, for which Perception Park is a partner.
