Caption: Neural networks can be trained with sample images supplemented with masks of the features to be detected
Michał Czardybon, general manager at Adaptive Vision, gives his view on how to make the most of deep learning in industrial vision
Deep learning is an exciting new technology, but there are still many misconceptions about how it is applied to machine vision applications. There are large neural networks in areas such as autonomous vehicles, customer behaviour analysis or computers playing Go, but here at Adaptive Vision it is used for industrial vision systems, with the biggest focus on quality inspection. In this article I will explain what can really be expected from deep learning for industrial inspection, as well as the system requirements to use it in projects.
Deep learning, in the context of image analysis, can be described as a powerful tool for recognising visual patterns. This might be, for example, a scratch on the surface of a metal part. In the past a sequence of image filtering, blob analysis, feature extraction and rule-based classification tools distinguished the relevant pattern from the background. The problem with this approach is that it requires a lot of engineering time and includes many parameters, which make the algorithms difficult to maintain. Today, things often become too complex to be hand-crafted by engineers; neural networks recognise these patterns in a very different way – by automatically learning from a pool of sample images.
The most typical application of deep learning in industrial vision is recognising if part of an image corresponds to a defect, such as a scratch, or – in a more general sense – to a feature, for example the edge of an object. In such a case the network is trained with sample images supplemented with masks of the features to be detected. It is called supervised training, because the expected outcome is demonstrated explicitly to the network.
Another kind of application is detecting any anomalies in the product under inspection. When it is impossible to define precisely what a defect might be or where it will be located on the image, then semi-unsupervised training is used. In this mode the network is trained on how ‘good’ the objects look. A trained network then analyses an input image and indicates if it looks typical – good – or not. This approach can be used, for example, to verify whether a packaged meal is complete and arranged correctly.
Other notable applications of deep learning tools include general object classification – for example, distinguishing between a chicken leg and a wing – and location, such as finding a part to be picked by a robot.
These are typical applications, but there is still much to say about what is needed in machines or production systems, in order to use neural networks. When talking about industrial inspection, my advice would be to use standard machine vision software, or libraries that include deep learning capabilities. Frameworks like TensorFlow or Caffe are very general and low-level tools that require a good deal of expertise. If one needs traditional template matching or barcode-reading tools, the user could write it, but as standard libraries are effective for all typical cases and extremely well optimised, this is not really justified, neither from a technical, nor an economic point of view.
In terms of hardware, a PC platform with a modern GPU card is necessary to train and run neural networks in a highly parallel way. There is also an option to use a standard PC, but only if the task can afford an image processing time of one second or more – on a GPU it might take 100ms for the same operation.
It is also worth noting that there are some significant differences between standard deep learning tools for industrial applications and the ones used in other markets. First of all, inspection applications usually work with as few as 20 to 50 sample images, provided that their quality is as good as in traditional vision systems. The more images the better, but because of the realities of the industrial vision market, vision engineers tend to rely heavily on image augmentations. These are special transformations that basically create thousands of artificial variations of the real samples.
On the other hand, an important characteristic of industrial deep learning systems is that they still need good lighting, the right lens and an industrial-grade camera to achieve 100 per cent reliability. There are some new application areas where the system can deal with more difficult imaging conditions – especially in automated agriculture – but this does not mean the user can stop thinking about the right hardware setup.
A network can be used to detect undefined defects, such as cracks in a Jaffa Cake
Also, don’t expect deep neural networks to detect defects which a human eye cannot see on the image. Neural networks might be faster, more repetitive and more accurate than humans, assuming limited time per inspection, but they must first be trained with samples manually labelled by us. What is more, the old rule still applies that if there is no information in the image, no algorithm will extract data from it, no matter how good it is.
Finally, I would like to emphasise that deep learning is not a thing of the future, but it is production-ready today. There might be some AI-related buzz in the media, which highlights complexity and concerns about the implications to our lives, but – at least in the field of industrial vision systems – it is simply a new approach to solve an old problem – there is nothing magical.
It is also clear that deep learning will not replace traditional machine vision tools completely. In some applications it is what is needed, but in many others it is only handling part of the task. Moreover, there are some applications, such as measurement or code reading systems, which are best handled with traditional tools.
In general, deep learning is most useful when defects, or other expected outcomes, are defined in an informal way and there is some inherent level of vagueness. Actually, despite wanting to have a 100 per cent well-defined problem, this is rarely the case in visual inspection. Customers and partners often talk about projects using phrases like ‘might be acceptable’ or ‘may vary due to production conditions’. Thus, instead of crafting and tuning complicated sequences of strict traditional tools that need maintenance whenever a tiny detail is changed, deep learning is used that adapts to a specific set of samples it is trained with – and it works with strikingly high reliability.
The whole process also has a big advantage in that it is inherently compliant with test-driven development. Start by clearly defining test cases and any ambiguity must be resolved with the customer up front. All these elements result in a very different way of building inspection systems.
Here, at Adaptive Vision, we strongly believe that neural networks come with so many benefits that they will dominate vision-based quality inspection systems, while at the same time opening a huge number of new applications that could not be solved with traditional tools.
