Following speaking at the AIA business conference in Orlando, Florida at the end of January, Jon Sobel, CEO and co-founder of Sight Machine, argues that with modern data analytics machine vision can be much more than just inspection
Machine vision is nothing short of remarkable. After 30 years of development, it is technology that modern manufacturing cannot do without. Ask many vision companies what they do and with justifiable pride they will point to vision’s widespread, growing use for inspection.
Yet from the vantage point of quality theory, inspection by itself is an inadequate goal. As Deming and Juran emphasise, inspection is a defensive act, a prayer at the end of the line. It is one thing to catch and cull a defect, but it is altogether another thing to know why the defect happened, how to prevent it, or better yet, when and why a process is starting to trend out of control. These needs extend well beyond a factory’s four walls. Of crucial importance is supplier quality: can I count on my suppliers? Do they have control of their process? How do I know?
The key to these insights is real-time data on the parts themselves, something that, to date, has been nearly impossible to achieve. But that is changing rapidly.
Industry giants such as Siemens, GE, Cisco and Google are investing massive sums in ‘Industry 4.0’, ‘the Industrial Internet’, the ‘Internet of Everything’, and ‘Android for Robotics’. And while these efforts’ precise contours have yet to be defined, the coming transformation is unmistakable. Modern factories will link devices to one another and focus relentlessly on data.
Common protocols, open standards and connected devices are moving onto the floor. As has happened in every other major field of computing, these elements will lead to an explosion of capabilities. While vision is one of a factory’s most complex technologies, as with all computing, image data is ultimately zeros and ones. Inevitably, our industry will move from hardware-centricity (what can this device do?) to a platform mindset (what can we do with the data?).
To achieve these capabilities, nothing new needs to be invented – the technologies to analyse vast quantities of data exist already. They’ve been hardened by billions of users on the consumer Internet. If we step back for a moment, it’s worth asking why we can't expect the same level of insight from our factories that we currently get on our phones?
What does ‘intelligent’ machine vision look like? From today’s vantage point, it’s hard to think about this because we accept that most cameras have limited storage. Consider what happens when, instead of discarding vision data, a factory can act in real time upon all the data generated by its cameras. Here are some examples:
Automatically generated controls, charts and alerts
The attached photos demonstrate how one fastener factory uses a $100 document scanner, vision algorithms, and Internet architecture to measure both the dimensions and grain flow of sampled parts, provide operator alerts, and generate control charts for operators – all from image data alone. If thresholds are violated, the operator receives an alert. Data for each sampled part is stored in the cloud and made available to factory customers, which gives this factory an edge in ensuring its reputation for quality. And if questions or disputes arise about parts after shipment, the factory can reanalyse the images along different parameters. All of this is done through a web browser.
Supplier quality
In this photo, we use vision techniques to analyse the ‘spheroidisation’ of steel provided to this same fastener company by its steel supplier.
The algorithms to analyse spheroidisation require less than 50 lines of open source software code (we use a framework we wrote called SimpleCV). And because supplier and customer can both review this data in their browsers, now both can independently assess the steel’s quality – each from their own systems. Internet architecture is the key – it allows for machine learning to improve the algorithms, remote updating of the software, and simultaneous views by supplier and client.
Compare images of parts as manufactured to customer data: where is the problem?
In another application under development, a wheel and tyre assembler stores photos of each assembly. If an end user reports a problem to the dealer, the dealer need only upload a photo of the assembly. By comparing the assembly when it left the factory to how it looks in the field, customer, dealer and supplier can understand what caused the problem.
In the automotive industry, disputes between the tiers can be bitter, often because they must be resolved in the absence of data. That will change.
Birth certificate for a vehicle
This application involves the analysis of more than 1,000 vehicle exteriors a day at a Detroit 3 final assembly plant. It associates data on part quality to each manufactured vehicle by VIN. As more and more checks are added, the manufacturer can track defect by type and by source.
These applications represent just the beginning of what’s possible when vision is combined with Big Data. Of all the sensors in a factory, vision is the richest source of insight. Once we in the industry move from seeing ourselves as selling inspection to enabling insight, whole new classes of opportunities will emerge.
--
Sight Machine, based in San Francisco, USA, provides a vision platform for the ‘Industrial Internet’. The software merges machine vision with Big Data, analysing data from sensors and cameras to assess industrial processes in real time.
