In an effort to improve operational efficiency and minimise or eradicate defects, a growing number of manufacturers in the automotive sector are introducing machine vision technology in quality control processes for components, modules, sub-assemblies and finished vehicles. So, what are the main current and potential applications of machine vision technology in automotive manufacturing quality control applications? What are likely to be the key innovations and trends in this area over the next few years? What role might vision technology play in the automotive factory of the future?
Visual Inspection
Ever since the birth of the automotive industry, manufacturers have worked tirelessly to improve productivity. However, as vehicles and components become ever more complex, companies across the sector have come to realise the significant limitations of relying on human vision for quality control – particularly in vision-related factory tasks like inspection, which require high speeds, constant operation and a lot of monotony. In an effort to face up to and overcome these limitations, a growing variety of machine vision technology has been developed to take over responsibility for visual inspection and control, as a key part of demanding industrial applications that require high-speed or high-magnification, often within 24-hour operational schedules or in manufacturing environments where repeatable measurements are required.
According to Marc Benevento, managing director at Industrial Market Insight, machine vision is also finding more and more applications in areas typically associated with a high cost of failure – particularly in relation to the ‘small, complex assemblies’ in engines and transmissions that are typically difficult to inspect with the naked eye, but where the cost of failure might commonly run into ‘thousands of dollars.’
‘Machine vision systems allow for 100 per cent inspection with reliability and cycle time improvements over human inspectors,’ he says.
Machine vision systems are also used widely to inspect adhesive beads and position parts on to vehicles in operations like bonding of the fixed front and rear glass. As Benevento explains, this means that the entire operation can be conducted without human installers. As structural adhesives are being used more frequently to impart significant performance attributes to the body in white, he also points out that more automakers are ‘considering the use of vision systems to ensure the proper placement of adhesives in critical to quality applications’.
Maintaining flexibility
One company that is actively involved in the application of machine vision technology to quality control tasks in the automotive manufacturing sector is Canadian electronic imaging company Teledyne Dalsa, which has created technology that uses a combination of camera sensors, processing hardware and software algorithms to automate complex or mundane visual inspection tasks and precisely guide handling equipment during product assembly.
As Dave Richardson, vision solutions manager at Teledyne Dalsa, explains, current applications of the company’s machine vision technologies include ‘positioning, identification, verification, measurement and flaw detection’.
‘Our machine vision solutions are deployed across all stages of automotive manufacturing, from body stamping and finishing, to powertrain and assembly, to final inspection,’ he says. ‘Tier one and tier two automotive suppliers use vision extensively to ensure that every part or sub-assembly is manufactured defect free to the OEM’s specification.’
In Richardson’s view, the main advantages of using machine vision in such applications are ‘the ability to meet throughput and ensure manufacturing quality’ – particularly in view of the fact that automotive subassemblies are ‘often produced in very high volumes, making human assembly and verification impractical’.
In addition, he highlights the fact that automotive suppliers have to maintain flexibility in their manufacturing processes to ‘accommodate small or large assembly changes associated with different makes and models of cars’.
‘For these demanding applications, machine vision systems are widely used to guide robots, track parts and verify every step of the evolving assembly process,’ he says.
Robotic grippers
According to Richardson, Teledyne Dalsa’s technology is already widely used in automotive manufacturing plants throughout the world – and he cites one particular application where a company uses the technology for guiding robotic grippers to lift automotive parts on to the assembly line automatically. If the robotic gripper fails to attach to a part in the right place, usually at designated marks, he says that the part ‘could tilt and be suspended from the robotic arm, or be dropped off the line’.
‘The automobile manufacturer tested several vision systems, and chose Teledyne Dalsa’s technology, which offered the high resolution and image quality needed to process images from two distinct assembly lines simultaneously. Triggered independently, our vision cameras take images, capturing multiple positions simultaneously to identify the type of vehicle part being moved, and then to ensure that the four robotic arms will grab the part in the correct locations,’ he says.
‘The flexibility to accommodate these two different types of inspections simultaneously is unique to our vision solution. By implementing this vision technology, productivity has increased six times overall and has reduced the potential for error significantly, protecting the safety of operators and eliminating the risk of damage to the vehicle part being processed,’ he adds.
Minimising defects
In another interesting recent development, the Indian-based automotive component company Timken India – which manufactures bearings, gear drives, belts, and chains – has introduced an automated inspection system at its facility in Bangalore that uses Teledyne Dalsa Sherlock software to enable more accurate quality inspection. The decision to introduce an automated system was prompted by a recognition of the limitations of the existing manual process – which selected random samples for inspection and defect identification in bearings – and the pressing need to introduce a more efficient way of inspecting complete batches of finished products before they were dispatched to customers.
In an effort to meet these requirements, the company has installed cutting-edge machine vision software that is specially configured to operate three cameras – two of which are positioned on each side of the manufacturing line, with a third with a clear view of the flat surface at the end of each roller bearing, in the process enabling a comprehensive 360-degree cylindrical surface inspection. In such a way, the new vision system is able to inspect the roller bearings and identify a number of potential defects – including anomalies such as unfinished end faces, missing plating and rust or tool marks, in addition to dents and scratches.
Another leading company in the field of machine vision technologies for quality inspection is German firm MVTec Software, which manufactures a large number of general purpose machine vision software products – in particular the flagship Halcon and Merlic systems – commonly employed for imaging and inspection functions across a wide range of sectors, ranging from medicine, surveillance and semiconductors, to optical quality control and metrology.
In terms of the automotive sector, both the Halcon and Merlic systems have been deployed in a number of production sites, where they are primarily focussed on the inspection of the array of components produced at manufacturing facilities, including simple tasks such as surface inspection, as well as more complex 3D measurements or pose estimation applications like robotic guidance.
Robotic guidance and tracking
In light of the large, and growing, number of uses for machine vision in quality inspection, it is, perhaps, a fair bet that the amount of automotive companies choosing to embrace such applications will rise over the next few years. Ultimately, Benevento predicts that machine vision systems could soon enable the robotic replacement of human assemblers in ‘any number of applications.’ In his view, this would provide a particular advantage in assembly operations that are dangerous or ergonomically inefficient.
‘The use of machine vision assisted robotics in these applications could reduce injuries and benefit workers and the bottom line,’ he says. ‘Also, as more multi-material bodies in white are introduced, machine vision systems could be used to inspect joint structure, fasteners and adhesives in areas where it is not practical for a human inspector to reach. This would ensure that quality is “built-in” to each body in white, rather than relying on intermittent tear downs to validate performance,’ he adds.
Over the next few years, Richardson predicts that machine vision core technologies will continue to be deployed across all aspects of automotive manufacturing for ‘error proofing, robotic guidance and tracking applications’ – with the trend in these areas increasingly on ‘improving ease-of-use and integration across the enterprise’.
Richardson also believes that deep learning technology will gradually emerge as a key innovation in automotive manufacturing quality control – particularly in view of the capability of deep learning to ‘comprehensively analyse’ and evaluate large amounts of data, also known as Big Data – as opposed to relying on task-specific algorithms
‘In machine vision applications, human inspectors define and verify individual features manually, whereas emerging technologies, such as deep learning, will offer manufacturers a trainable solution for applications that are less well defined. Deep learning uses software-based neural networks, which mimics the pattern recognition abilities of human intelligence to train on certain features of the object such as colour, shape, texture and surface structure,’ he says.
‘Deep learning technology is most suitable for demanding OCR applications, locating complex surface and cosmetic defects such as scratches and dents on turned, shiny or brushed parts and identifying tiny paint defects that are not visible to the naked eye,’ he adds.
Richardson also argues that machine vision, in particular smart cameras, are a key technology for the factory of the future – largely because such devices are ‘visible on the factory network and can be easily reconfigured at a moment’s notice. Vision will provide critical functionality in smart factories as it can be used not only to control quality, but to guide or communicate change to downstream devices’.
Another interesting application of machine vision technology in the automotive sector is for the variety of tasks that could be categorised as ‘human assistance’.
One company actively involved in this area is German firm Stemmer Imaging, which is soon to launch its new SC-10 human assistance smart camera system, aimed specifically at the manual assembly market.
According to Mark Williamson, managing director at Stemmer Imaging, although other systems focused on this market have been available for some years, they have suffered from the fact that they are ‘expensive and need dedicated integrator expertise to deploy’.
In seeking to address these limitations, he claims that the main advantage of using the new human assistance camera is the fact that it combines the delivery of work instructions to a screen by the operator, with hints relating to where components need to be placed – in the process validating the fact that the components are fitted in place ‘and recording that all assembly steps have been completed’.
‘This replaces PC-based and paper-based instructions and paper inspection check lists and ensures all assembly steps are completed. Normal vision systems and smart cameras are aimed at automated lines and give no work step guidance to the operator. This camera combines instructions with the checking. It is unique as a simple camera,’ he says.
As more and more tasks and functions related to assembly and production become automated, Williamson predicts that the use of machine vision systems for quality control will continue to grow – and he points to the fact that set-ups such as Stemmer’s own LMI 3D vision systems and Teledyne Dalsa’s 2D vision systems already enjoy ‘good market penetration’.
‘Saying that, manual assembly is still significant. With the demand for improved quality, automating the inspection and validation of manual inspection will grow significantly. Our new camera system reduces the cost as much as 75 per cent over traditional systems, by removing the need for an integrator.
‘Evidence shows this enables a rapid increase in further adoption,’ he says.
