Next month, from 9 to 13 September, leaders in the field of computer vision will gather in Bristol, UK to discuss the latest advances in image processing algorithms for the annual and confusingly titled British Machine Vision Conference (BMVC). Definitions of machine vision and computer vision vary, but generally, and for the purposes of this article, machine vision refers to industrial imaging and factory automation – whereas computer vision spans a wider remit and is concerned with any academic research that aims to extract information from images.
‘Anything that purely stops at the level of manipulating the image isn’t really machine vision; you’ve got to have the machine [for it to be machine vision],’ states Neil Thacker, senior lecturer at the University of Manchester and BMVA Secretary. The British Machine Vision Association (BMVA) organises the BMVC.
So how closely are the two disciplines linked, and can they learn anything from each other? The amount of crossover between machine and computer vision differs depending on who you talk to. Machine vision algorithms in software libraries like Halcon from MVTec, Stemmer Imaging’s Common Vision Blox, or the Matrox Imaging Library all originated at some point from academia, but Dr Thacker feels there’s less interaction than there was in the past. ‘Academics do computer vision and industrialists do machine vision. It was less that way 20 years ago,’ he says.
According to Professor Carsten Steger, head of MVTec’s R&D department and adjunct professor at the department of computer science at the Technical University of Munich, there still is ‘substantial overlap’ between the two areas. ‘We [at MVTec] are constantly looking for interesting developments being made in computer vision that we can apply to Halcon,’ he says. ‘If we find something interesting then we have to make it work in industry. That’s sometimes not the focus of computer vision research.’
MVTec has good connections with the Technical University of Munich and it sponsors PhD students at the University to carry out research for the company.
One of the problems of making better use of academic research lies in turning that research into a sellable product. Mark Williamson, director of corporate development at Stemmer Imaging, says there is a big difference between working on research for an algorithm and commercialising that algorithm as a product. ‘From the industrial side there is a huge amount of work that has to be done to turn research into a product. It’s not just the algorithm, it is making it a robust product with technical support, trialling it in the field – there’s a lot more before you can introduce it as a product.’
Stemmer Imaging has links with academia and is involved with the BMVC. Some of Stemmer’s Common Vision Blox (CVB) tools were also initially developed in academic research – including CVB Flex Inspect, which deals with acceptable variations in webs of material like textiles or print runs, as well as its image stabilisation and optical flow CVB tools, which were released within the last year.
However, Williamson still feels there is a gap between industry and academia: ‘I’m quite disappointed by the lack of transfer of research knowledge into the commercial world,’ he says. ‘There still seems to be this massive void of information from academia that doesn’t flow into industry.’
Williamson suggests one way of narrowing the gap is by having more PhDs sponsored by industry. Stemmer sponsors PhD students in Germany, working on projects that might eventually become part of CVB. ‘I think there would be a good amount of interest from the universities in the UK for schemes like this,’ he says. ‘Students need to learn how to take their algorithms further to make them easy to adopt by industry,’ Williamson adds. ‘Many researchers don’t deal with "what if this doesn’t happen"; they prove that it works in a perfect environment, but not when conditions are less than ideal.’
The power of good engineering
One of the reasons for the gap between industry and academia might be because machine vision doesn’t need computer vision algorithms to a certain extent. It comes back to the ‘machine’ aspect of machine vision, in that there is an awful lot of engineering that goes into building the machinery, which can quite often drastically simplify any vision task before image processing is even considered. Standard manipulation and lighting rigs can make the problem as simple as possible for the vision task.
‘A good machine vision system might need only to be simple in order to do its job,’ comments Dr Thacker, ‘but a good computer vision system rarely can be simple in order to do its job. If you can make your vision problem simpler for yourself by careful control of lighting and careful control of your machinery, you can end up with really quite simple image processing problems. In order to think that computer vision can be a benefit [to machine vision], you’ve got to find a situation in your machine vision problem where you can’t engineer the problem away. That’s actually quite difficult. An awful lot of problems can be engineered away.’
According to Dr Thacker, ultimately, computer vision is inspired by ideas of biological perception and human sight. He says that is the complete opposite to engineering the problem away and that, in computer vision, researchers want to solve problems as they are – despite having no control over the incoming data.
Dr Thacker is working on a project to analyse satellite images of the surface of Mars. The project uses pattern recognition techniques to try to recognise textures and to automatically calculate the number of different terrains present in particular areas. Dr Thacker says that, at this point, the team doesn’t even know what science they can do with the data, but they have a belief that they’ll be able to answer interesting scientific questions later on.
Areas where machine vision might look to computer vision to provide answers are any imaging tasks with unconstrained lighting. Outdoor applications are one example, such as the image processing for driver assistance sensors being developed for cars, or for mobile robots that have to operate in ambient lighting.
Funding is one potential problem for researchers working on industrial applications according to Dr Thacker, caused, in part, by manufacturing moving from Europe to the Pacific rim and China and Japan.
Dr Thacker feels that there isn’t the need for robotic manufacturing cells in Europe anymore. ‘Without the perceived need, there’s no impetus to work on it [industrial problems] and there’s no funding to work on it, and academics go where the funding is. If you wanted to get a grant from the Engineering and Physical Sciences Research Council (EPSRC) to build an automated assembly robot based on vision you would never get the money. If you wanted to do something like that, you’d need an industrial sponsor.’
At the same time, robotics is less of a focus for computer vision than it was. ‘They [computer vision scientists] are looking at anything you can do with images rather than just interacting with robots,’ Dr Thacker says. Computer vision has moved in directions where it’s doing things like automatically filtering images online or calculating camera transformations for graphics rendering in Hollywood films.
How academics can benefit from industry
Of course, not all computer vision should be targeted at a particular application and fundamental research is a big part of the academic world. However, machine vision provides real-world image processing requirements, such as possible image degradations in an application or how much bandwidth is needed. ‘Scientists often work with good-quality images, whereas, in industry, the images are often of a much lower quality,’ explains Bruno Ménard, image processing group leader at Teledyne Dalsa. Machine vision algorithms have to be extremely robust to cope with variations in the image.
Teledyne Dalsa provides its Sapera and Sherlock machine vision software suites, along with other image processing packages. Ménard points out that a lot of Teledyne Dalsa’s high-level algorithms such as those for pattern matching, barcode reading, and OCR are based on developments in academic image processing. The software installed in Teledyne Dalsa’s smart cameras self-adjusts its parameter settings according to the images captured, a function that originates from the scientific side of image processing.
Professor Steger at MVTec comments that, when the company investigates potential new algorithms for Halcon, MVTec engineers look at algorithms that are interesting in principle and those that would be useful in practice, taking into consideration run time and robustness. ‘Does it work on more applications than the author has put forth in his paper? This is one question we ask,’ he says.
Speed is also an important factor. ‘If you have an algorithm that runs for six hours it’s nice if it produces good results but it’s impractical in industry,’ comments Professor Steger. Ménard agrees, commenting: ‘I would like to see more work on developing faster versions of algorithms. Speed is the first requirement for machine vision; when you don’t have speed you can’t fulfil other fundamental requirements.
High-level algorithms like OCR in Teledyne Dalsa's Sherlock machine vision software are based on developments in academic image processing. Credit: Teledyne Dalsa
‘Machine vision shouldn’t have to rely solely on fast computers, but also work with well-designed, robust algorithms,’ Ménard continues. ‘When engineers set up image processing tasks on the factory floor they can’t re-engineer the production line to suit the algorithm.’
Computer vision is about training people
Regardless of whether there are any formal links between the two subjects, Dr Thacker believes people who have training in the one will crop up solving problems in the other. ‘We get a very large throughput of people who come in, do PhDs and then disappear off into industry,’ he says.
‘You will get a bleed through of computer vision ideas in industry eventually.’
According to Dr Thacker, conventional scientific disciplines like physics will train people to be academics whereby they’ll stay in academia and pass that information on to the next generation. Computer vision, on the other hand, is really more about training throughput than training for academia. Inevitably, he says, a lot of the developments in computer vision will show up in other areas, purely because the scientists take that knowledge with them into industry. ‘I don’t know whether you really need a formal link between the two subjects for success in the one field to move across to the other,’ he adds.
‘My prediction would be that, over time, better statistical methods will be dropping out of areas like computer vision and will be picked up by areas like machine vision as improved ways to configure and calibrate systems,’ Dr Thacker states.
One of the reasons for a lack of crossover between the two disciplines, according to Dr Thacker, is that some academics feel that machine vision is solved and it’s a case of using the existing tools. ‘I don’t believe that,’ he says. ‘I still think there are an awful lot of challenges in industry and that there is knowledge that hasn’t moved across into industry.
‘I feel sad, to some extent, that there isn’t a better link between more hard-nosed industrial applications and some of things going on in academia. But to have a better link the funding would have to be there – and, at the moment, it just isn’t,’ Dr Thacker continues, though he adds that, in some countries, it is easier to gain funding. He gives examples of America funding research through DARPA, which feeds directly into industry, and the Japanese funding research on robotics and developing robots like ASIMO.
‘If you ask have we managed to build anything even comparable [to ASIMO] in this country, the answer is no because the funding hasn’t been there. The intellectual ideas are there and there’s certainly a lot that industrialists could get from academics if the linkup could be made, but you’ve got to have the infrastructures to do it and they’re not there.
‘If you ask, is the average academic going to spend time phoning around to find industrialists to talk to about funding research, the answer is probably no, at least not in the UK. Here, it’s not the way it works – and it shouldn’t be.’
Stronger links with academia will be driven, in part, by applications that have a need for better image processing. 3D imaging is a more recent example of computer vision research that has found its way into industry. If machine vision companies need to solve specific problems, then schemes in collaboration with universities could be one way of doing this.
