In the spring earlier this year, I helped sow a meadow with a mix of wild flower and grass seed in the hope that, eventually, the flowers and grasses would outcompete the thistles. Success so far has been limited. One of the better areas was a stand of linseed, but instead of the expected, more traditional blue flowers, these plants turned out to have white blossom.
Mistakenly growing a variety of linseed with white flowers rather than blue is not the end of the world but, for seed producers, the plant variety that is packaged and sold to growers matters, sometimes down to the level of the single seed. According to Jens Michael Carstensen, CEO of imaging firm Videometer, which works heavily in the seed industry, it can be required by law that batches of seed are sufficiently pure to be traded.
Videometer provides hyperspectral imaging systems and generates almost half its revenue from systems sold for seed and grain analysis. (Inspecting other foodstuffs accounts for a quarter of its revenue, with the remaining quarter for solutions for pharmaceutical inspection, medical devices, cosmetics and other areas.)
For the kind of highly accurate analysis where each and every seed is sorted, Videometer would use a robot with validated picking, Carstensen said, ‘so we are sure not only that we detect a seed that’s the wrong variety, but that we also remove it.’ The aim is to have validation throughout the process chain.
‘A lot of effort is put into breeding the right kind of seed,’ Carstensen said. ‘By the same token, a lot of effort goes into making sure the supply chain works so that the seed sown is as good as it’s supposed to be, it’s the correct variety, and it germinates and grows into a strong plant.’
Spectral imaging can help in both production quality control and in plant breeding – two important parts of the seed industry. In plant breeding, the aim is to screen seeds to pick out candidates with the right traits.
In either breeding or production, spectral imaging can determine the variety and whether there are off-types contaminating a batch of seed, such as seeds from different crops or weed seeds. ‘Purity is extremely important in seed production – that it’s the right crop and the right plant variety,’ Carstensen said.
Spectral imaging can also show whether the seed is damaged in any way, from insects or fungi, for instance, and whether the seed is likely to germinate. ‘Saying whether a seed will germinate or not is tricky because normally you need to sow the seed and wait for it to germinate and grow,’ Carstensen explained. He said that the Holy Grail would be to detect whether a seed is viable or not with imaging, something that’s not possible at the moment, but spectral imaging can give indicators as to viability.
‘We don’t expect that, for all seeds, we can tell the entire germination story based on the dry seed, but for many seed types we can see how viable they are and how likely they are to germinate,’ Carstensen continued. ‘Seed suppliers can make decisions based on a partial knowledge of germination, and that will help them in their logistics.’
Along with pest damage, spectral imaging can assess whether a seed has been harvested immaturely – a chlorophyll fluorescence test can show how much chlorophyll there is in the seed and therefore its maturity – and also whether a seed has been stored for too long by identifying oxidative damage.
Spectral imaging can show a lot, but nevertheless, the organisations controlling seed testing – the International Seed Testing Association and the Association of Official Seed Analysts – require a germination test.
Videometer would typically use its standard system, VideometerLab, for seed analysis. This scans across a spectrum from 350nm to 1,000nm to create a data cube. ‘We can go a little further into the deep UV and the near infrared, but there are challenges getting super apochromatic lenses to cover the complete spectrum,’ Carstensen said.
The company has a machine learning engine that it trains with spectral imaging data from different crops and plants. The algorithms look for different markers in the seed, such as the variety or whether there’s fungal damage.
‘We work with all crops,’ Carstensen said. ‘We look for similar markers with spectral imaging, but each crop will be different and it’s difficult to extrapolate from one crop to another. We work a lot with vegetable seeds and field crops – wheat, barley, rice, oil seed rape – but also flower seeds and tree seeds.’
- Greg Blackman speaks to machine builder Bühler about how optical sensing can maximise yield in grain processing
- Farmers are starting to reap the rewards of robotics and machine vision, as Keely Portway finds out
Videometer began 20 years ago providing spectral imaging for very accurate colour determination in production – sorting based on very subtle nuances of colour.
‘Spectral imaging is a mature technology,’ Carstensen said, ‘but it’s something that’s specialised in terms of who is providing it, because the data is massive and you have a lot of opportunity to get noise instead of signal.
‘Spectral imaging cameras and components are only part of the solution,’ he continued. ‘A true solution for the industry would have to integrate a lot of components: illumination sources and illumination geometry, cameras, controlling electronics, machine learning – putting all this together is quite a multidisciplinary undertaking.’
Videometer only uses LEDs and has done so for 20 years. Its systems combine a number of LEDs emitting at specific wavelengths throughout the UV, visible and near infrared range. The LEDs are strobed in an integrating sphere extremely quickly so that the system captures each image under a different wavelength of light, acquiring images as fast as the camera allows. ‘Typically, we would take 20 wavelengths within half a second, calculate the data cube and analyse it to provide the result,’ Carstensen explained.
He said that a lot has been made possible by the increase in computer speed – deep learning is one example. He also said that the power-over-price in LEDs has improved, especially in the UV, where it’s now becoming feasible to do quite sophisticated measurements that weren’t possible in the past because powerful deep UV LEDs used to be prohibitively expensive.
Videometer's spectral imaging solution for analysing seed
‘We definitely believe that UV is important for spectral imaging,’ Carstensen said. ‘The challenge is that normal optics will not go to a shorter wavelength than 400nm, sometimes 350nm. You need specialised optics that are more expensive, and still hard to get – there’s not a lot of choice when you go down to UV optics. You can get them, but if you have specific requirements within the UV range you run out of suppliers very quickly.’
Separating the wheat from the chaff
The seed companies that Videometer works with and that have been discussed so far are producing seed that will be sown to grow into a plant. But what about grain harvested for food? The Swiss R&D organisation, CSEM, has worked with machine builder, Bühler, to improve quality control on one of its milling machines.
Sébastien Blanc, senior R&D engineer at CSEM, spoke about the project at the Vision show in Stuttgart in October. The problem Bühler presented CSEM with was to reduce the error rate when measuring the mass flow of grains passing through its milling machines. Bühler’s machines typically use a deflecting plate in the shoot to measure the force of grain passing down it and from that calculate the mass flow.
Without an accurate mass flow measurement, grains of different quality get mixed together – Bühler was registering error rates of up to 8 per cent in mass flow, which can equate to around 200kg of grain per hour at the speeds at which these machines operate.
Blanc explained that Bühler wanted to lower the error rate to less than 3 per cent for grain moving at a speed of 1.5m/s. He said, on top of that, Bühler needed something that is compact, cheap and with low compute resources.
Blanc and the team at CSEM developed a vision system with embedded AI able to achieve an error rate of less than 3 per cent. The images could also be used to add a layer of quality control to the process.
The CSEM system uses a 1.5-megapixel RGB camera running at 60fps. This is paired with custom 150W illumination that can deliver 5µs pulses to avoid image blur. ‘We used an Nvidia Jetson Nano to orchestrate all the peripherals and run the algorithm in real time,’ Blanc explained during the trade fair. Everything is processed within the vision system. The team was also able to include a small web server running onboard, to which mobile devices can connect wirelessly.
Blanc said in his presentation: ‘We developed a flow speed algorithm to measure the speed of the grain. We can get an accuracy of 40µm at a speed of 2m/s. This is thanks to the help of the powerful LEDs.’
Bühler is not interested in flow speed, however, it is interested in the mass flow in kilograms per second. This can be calculated from the flow speed multiplied by the cross section of the shoot, multiplied by the density of the grains, which depends on the water content of the grain. The vision system doesn’t give the grain density, so the camera is combined with Bühler’s standard mass flow measurement system to give real-time analysis.
CSEM also developed a neural network to detect grain type, using the saturation and brightness information in the images to identify the grain.
‘We wanted to have an algorithm that is able to assess the quality of the grain, and for that we needed a way to segment the grains,’ Blanc told the audience in Stuttgart. CSEM used weakly supervised training of its neural network to segment the grains.
‘A data-driven approach [to AI] requires a lot of annotated data, which is sometimes cumbersome and hard to get,’ he said. The team therefore annotated a single image of the grain, and then went into a process of data augmentation, cropping the single image into a lot of sub-images. ‘We did a lot of stretching, rotating and flipping to generate a few thousand images that were used to train, test and validate a neural network,’ he said. ‘As an input into our neural network, we can provide 32 cropped images and as an output we get 32 masks of a single grain of segmentation,’ he said. ‘This is useful to fit an ellipse on these masks.’
A single image of wheat, for instance, was iterated three times, so 3 x 32 samples, which provides 86 valid grains on top of which ellipses were fitted. From the ellipses the group extracted the major and minor axes to make some statistical analyses and get an indication of grain quality.
‘If it’s not necessary, we don’t push to go into the data-driven approach,’ Blanc said. ‘The algorithm to detect the type of grain is based on classical image processing.’
Bühler uses lots of optical and imaging methods in its grain sorting machines to squeeze the maximum amount of usable grain from a harvest. The percentage gains from optical technologies can be quite small, but because these machines are sorting huge amounts of grain per hour, the benefits for its customers can be massive. It also helps reduce waste in a food system that’s getting pushed to the limit in terms of feeding the planet.
