Compared to the history of agriculture and viticulture, machine vision has been around for a heartbeat – and machine vision in agriculture a mere cardiac flutter. The culture of wine growing is an ancient one with special emphasis on place and what the characteristics of a place – the soil, the climate, the topography of the land – bestow on a wine. The concept is known as terroir, from the French word terre meaning ‘land’ and is the basis for characterising wines according to geographical region. A classic example is the Grand Cru vineyards in Burgundy, which were originally established by Benedictine and Cistercian monks, who, because of their vast land holdings in the region, were able to delineate plots of land producing exceptional wine.
The culture of wine growing and terroir is held in high esteem, but new utterly modern methods stemming from precision agriculture are also finding their way into the practice of growing vines. Precision agriculture, a farming management method that aims to improve yield by responding to intra-field variations, was initially implemented on large farms in Australia and the US, but is now being applied in South Africa, Chile and Europe. It is based on satellite imaging and GPS, but other tools like camera sensors are also now available to farmers to help them better manage their land.
One project being undertaken by the Polytechnic University of Valencia in Spain is to use imaging to estimate vine vigour and correlate it with yield. The researchers want to develop the system as a tool to give farmers more accurate information on when to harvest the grapes.
Francisco Rovira Más is leading the project at the Polytechnic University of Valencia: ‘We’re trying to make wineries in Europe more cost efficient. European countries are competing with other countries where labour costs are lower and wine costs less to produce. One of the ways to improve the cost efficiency of wine production is to introduce new technologies.’
As wine is a speciality, high-value crop, as opposed to crops like wheat, which is a necessity crop, growers are more likely to employ new technologies like machine vision, according to Rovira Más.
The concept of precision farming has been applied with some success in the Corn Belt in the US, mainly with bulk crops like corn, wheat, and soybeans. One of the problems though with introducing novel technology to farming, according to Rovira Más, is that a lot of it hasn’t reached actual farmers yet. ‘There is a big gap – the technology is being developed by universities and research institutes, most of the time by PhD students, and when the researchers finish a project most of the solutions die and never make it to the market. This is largely because of the technology’s complexity, price, and the difficulty of managing all this information.’
The Polytechnic University of Valencia’s system comprises one GPS receiver, a computer, and one imaging sensor, a monochrome camera from JAI. The researchers wanted to keep their system simple and so avoided implementing things like multispectral sensors and thermographic cameras, which would give useful data, but would raise the complexity to a point where the system would never be commercialised in the short run.
‘Speaking with farmers, they say they have problems in determining when to harvest grapes to produce high quality wine,’ explains Rovira Más. ‘Farmers say that many times they don’t know what they’re going to get in terms of quality.’ At harvesting time, typically in October, there is a lot of rain and the window for harvesting is quite short. A lot of the time all the grapes are mixed together, which means the quality is unknown. The imaging system provides the farmer with information on the vigour of the grapevines prior to harvesting.
The JAI camera is a monochrome device detecting from the ultraviolet into the near infrared. UV reflectance hasn’t been well explored in agriculture, according to Rovira Más, so the team are investigating that bandwidth. NIR is good for vegetation because reflectance is high. The results in the NIR are promising for detecting vigour, he says.
The system is installed onboard a modified tractor. The vines are mapped during June and July when the grapes are changing colour. At this time of year in Spain, the temperature in the field is very hot, which the camera has to be protected against. In addition, cameras that pass through vegetation can be broken or the cabling damaged if they are not optimally mounted and located in the vehicle. But the main problem, according to Rovira Más, is with illumination and the strength of the July sun casting shadows. There can be very dark and very bright pixels in one image, which complicates the extraction of key information from saturated areas.
An imaging system developed by researchers at the Polytechnic University of Valencia is being used to determine vigour of vine plants, with an aim to correlate that with yield. Credit: Polytechnic University of Valencia
The system uses Edmund Optics’ lenses and the research team was one of the winners of Edmund Optics’ higher education programme in 2011. ‘We wanted to use reliable hardware, including camera, GPS, and good filters from Edmund Optics to develop a high quality system,’ states Rovira Más. Edmund Optics’ NIR filters were used to separate the NIR reflectance from vegetation and to block the other wavebands. The researchers have also developed algorithms for quantifying the vegetation in real time as the tractor is moving. The images should give data on how vegetation is changing and relate vegetation changes with yield changes.
‘We’ve seen good correlation between vigour changes and yield, although we want to carry out more testing,’ Rovira Más comments. ‘Some work from Australia has suggested that the relationship between vine vigour and yield is not always straightforward so we want to investigate further, possibly through mapping other parameters as well.’
Elsewhere, precision agriculture techniques are being applied in Washington State in the US, where apple growers are interested in implementing robotics in the field. The producers are even changing the way they grow apples and using a trellis, similar to grape vines. ‘They are changing processes that have been working for years in order to adapt to new technology,’ says Rovira Más. ‘We also see this trend with olive trees in Spain – the spacing between trees used to be huge and it used to take seven or eight years to get to full production. Now, olive trees are planted very close together, using irrigation, and getting crops in four or five years.’
Water is also a well-correlated indicator of yield and a map of moisture in the soil is something that farmers could take into account
when they predict yields. ‘We face a trade-off; we’d like to get better results and more robust systems, but at the same time we’re trying to simplify the system and introduce it to the farmers,’ continues Rovira Más. ‘The detection and control of water stress in the leaves seems to be very important in the production of quality wines. Thermographic cameras are coming down in price, which will allow us to measure this, but at the same time we don’t want to complicate the system. The other big problem is reliability – computers sometimes fail and GPS is sometimes blocked. We want to make our system more robust and reliable and try to keep it simple.’
Cherry grading
Imaging systems are finding their way into the field, but they are still largely research orientated with very few systems commercialised. Where vision plays a much larger role is in factories sorting and grading crops like apples or cherries once they’ve been harvested.
‘Most crops nowadays use vision systems for sorting in some respect,’ states Roland Scheffer R&D manager at Ellips, a Dutch company manufacturing machines for grading and sorting fruit and vegetables.
Ellips is in the process of installing a vision system at a factory in the US sorting cherries. The site currently has 2,400 people sorting the fruit. ‘If you can reduce that workforce by even half it’s a lot of money saved,’ he says, stating that the expense of manual labour is one reason for automating the process.
Inspecting cherries requires high-speed imaging, as the sorting machines run at around 30 cherries per second. This occurs on at least 10 lanes, equating to a rate of inspection of 300 cherries per second. Ellips machines scan the cherries with high-resolution cameras from Point Grey. Ellips has developed its own operating system and algorithms to be able to analyse the images at those speeds.
The system analyses the image to identify the contour of the cherry. It uses three cameras, one infrared, one colour, and a third monochrome version incorporating a filter to improve the contrast of defects, imaging in the infrared to do this. The three cameras inspect one lane, with each capturing 10 images per cherry (30 high resolution images in total) to cover the fruit’s entire surface area as it rotates.
‘The colour and infrared cameras are synchronised and the images overlaid onto each other to provide more information regarding blemishes or stem areas,’ explains Scheffer.
The infrared image identifies the contour of the fruit – cherries can be very dark, but when viewed in infrared they appear light and can be picked out well against the background. ‘Contour information is used to locate the stem, from which the system can determine the orientation of the fruit and measure its diameter at its widest part along the shoulder,’ Scheffer continues. The cherry’s contour is also used in the overlain colour image to calculate its area. This is important when determining the percentage coverage of a spot or blemish with regards to the total surface area.
The fruit are also graded by colour, in multiple classes ranging from light to dark. ‘We also inspect spots on the cherry, soft shoulders, cracks, if the cherry is stemless or not,’ Scheffer says. Producers want to grade the fruit and also sort out spurs, small cherries attached to other cherries.
‘We need a lot of detail in the images since the cherries are small and when you’re looking for defects you need high resolution,’ Scheffer states. ‘The Point Grey cameras provide the resolution necessary for sorting the fruit.’
The requirements are getting more specific and there are higher market demands which have to be met, according to Scheffer. He comments: ‘Cherries especially are difficult to inspect because the stems fly all over the place and can enter into an image on top of another cherry, which complicates the analysis. The algorithms will determine which stem belongs to which cherry and discard confusing images. Because it’s rotating, every image is different and all need to be checked.’
The cherries are classified, directed into different lanes with air jets and are sorted further before arriving at the packing area.
As cherries are a natural product, Scheffer says that it’s difficult to give percentage accuracies of the sorting. However, to give an idea, at the site in the US, the factory is currently using mechanical sizing systems, which have an accuracy of 40-50 per cent. ‘Our optical system is at least twice as good for sizing the product,’ states Scheffer, ‘it provides somewhere between 80-90 per cent accuracy.’ Also, he adds, the company weren’t able to do colour sorting – colour and defect sorting were being done manually up until now with a lot of people.
