The Harvesters, an oil painting by the Dutch master Pieter Bruegel the Elder, was often a teaching aid when I was a student to illustrate how crop breeding has changed agriculture. The picture depicts labourers bringing in sheaths of wheat in 16th century Flanders. It’s not the farming practices that we were asked to look at, but the height of the wheat, which reached the farm hands’ shoulders in 1565 when the picture was painted – walk through a field of wheat today and it might be around your knees.
The modern dwarf varieties of wheat are the result of many years of plant breeding. A stocky plant isn’t flattened by high winds or heavy rain, while at the same time more of the plant’s energy goes into the grain, rather than into growing tall.
Other characteristics can also be enhanced by breeding, such as yield, grain quality and disease resistance. Now the National Physical Laboratory (NPL) in the UK is offering plant breeders and agricultural researchers new measurement methods based on imaging that have the potential to accelerate plant breeding trials, and ultimately improve food security as farmers seek to feed a growing population.
The NPL has built a wheeled platform equipped with a number of 3D imaging technologies, which can be attached to a tractor or otherwise pulled across a field. One version of the imaging rig includes an 18 stereo camera system. The intention is to generate a complete 3D scan of a field trial plot to assess how varieties of wheat grow.
NPL researchers also use time-of-flight, lidar, hyperspectral imaging and other cameras, including Photoneo’s 3D structured light camera supplied by Multipix Imaging, which is Photoneo’s distributor in the UK.
‘Currently, it takes in the region of five to ten years to develop a new variety of plant because of the throughput issues,’ explained Dr Richard Dudley, science area leader for electromagnetics and precision agriculture at the National Physical Laboratory. ‘With more data we feel we can reduce that – maybe down to a couple of years, a couple of seasons.’
Data from a 3D scan can give seed manufacturers an early indication of what varieties they want to produce for next year. ‘It can give them [seed producers] a three-month advantage, in time to decide what they are going to upscale production of, and that can be really valuable,’ added Dudley.
NPL is focusing its efforts on wheat initially, because it’s one of the highest value crops grown and bred in the UK.
Plant breeding requires lots of physical measurements, according to Dudley, which traditionally are made by researchers walking around the plot with a ruler and taking photos. NPL’s imaging platform can capture crop trials in a lot more detail in 3D. Algorithms are then applied, to give crop breeders more data to make decisions.
NPL will be going out to measure field plots in June
NPL can provide data on how wheat is distributed in the field, along with the ability to zoom into the 3D point cloud to view individual plants.
‘The data is more accurate, and they’ve also got year-on-year comparisons, rather than ruler measurements and pictures which don’t often tell them everything they need to know,’ Dudley said.
Image capture
The minimum area of land the NPL is asked to measure is 3 x 2 metres. This is captured at millimetre resolution, volumetrically across about half a metre. ‘It’s quite a big volume and quite a high resolution, and you can’t spend too long because things like wind and movement in the crop get in the way. So you need to capture it relatively quickly,’ explained Dudley.
Time-of-flight imaging is reasonably fast, offering 24fps or more, according to Dudley, but these cameras often don’t have the resolution required. The Photoneo camera, an area-scan device operating by what the Slovakian firm calls parallel structured light, has higher resolution and can capture movement in 3D – the Phoxi XL camera captures up to 3.2 million 3D points for each scan, at 16 million points per second throughput – although it still doesn’t meet all the demands of NPL.
NPL's imaging rig can scan wheat at millimetre resolution
To cover a 3 x 2-metre plot, NPL would use up to 18 cameras, possibly including three Photoneo scanners. The point clouds from these cameras are then combined to give an image from different angles. Which imaging technique is used comes down to the area of land covered, the speed of capture, and the resolution required.
There are difficulties in overlapping the images and getting the complete view, Dudley said, and errors can be produced when combining point clouds.
The measurements are made early in the day or late in the evening when the sun is low in the sky, because practically every camera technology used struggles with direct sunlight, according to Dudley. ‘I still think there’s a long way to go on most scanners out there in terms of [operating in] daylight,’ he said. ‘Midday, bright sunny day, most systems still fail.’
He added that lidar is the only technology that can just about deal with bright sunlight, but that lidar has shortcomings for the NPL team in other ways.
Making sense of the data
NPL is developing algorithms that fit to individual ears of wheat. The algorithms first identify where the ears are, then calculate the volume of each ear along with other aspects like length and number of grains.
‘It’s not trivial; it’s pretty tough stuff,’ Dudley said. ‘Fitting shapes into complex point clouds, although it’s been done a bit for autonomous vehicles, it’s still pretty challenging.’
The researchers are taking two approaches to analysing the data. The first is clustering the 3D points using a combination of MVTec’s Halcon image processing software, also supplied by Multipix Imaging, and Matlab vision algorithms. These look for densities of points and try to separate noise and stems, which are generally thinner than the ears. Shapes that resemble an ear of wheat are then fitted to find the ears in the point cloud.
The other approach is based on machine learning. The algorithm learns what an ear of wheat looks like from classified ears, then hunts them in the point cloud. But these shapes aren’t individual objects sitting with nothing around them; often the ears are overlapping or touching, which creates strange double-headed shapes.
Dr Richard Dudley, science area leader for electromagnetics and precision agriculture at NPL
Dudley said that the team is starting to get up to 90 per cent identification in an image of around 100 to 200 ears of wheat. The data produced after this pre-processing is then much higher quality.
At the moment, the NPL team captures the data in the field and then post-processes it in the lab. That can be done within a day. However, the team is also starting to work with groups that have 5G capability out in the field. It then becomes possible to do some cloud computing on these large datasets.
‘This is much more [relating to] commercial farming, where you might want to make decisions on the vehicle that’s out in the field,’ Dudley said. ‘That’s the future, trying to do data processing on-the-fly and have feedback as a vehicle is progressing through the field. The 5G network, with its capability of sending high data rates, is certainly something we’re looking at using.’
Food security
Dudley made the point that, currently, global production of wheat is struggling. ‘It’s difficult to produce as much grain from the area of land we’ve got, so it’s really important to develop these new varieties of crop that yield more, so we’ve got food for the future. Wheat is one of the main staples in our diet; it accounts for a lot of calories and protein,’ he continued. ‘That’s the driver, both economic and sustainability for the future.’
NPL is trying to secure its first customers for next season, starting in June. It will be going out to customers and measuring their field plots.
Dudley said that he is still looking for new imaging technologies to capture these 3D plots, both faster and in higher resolution, and with lower sensitivity to sunlight.
