The stringent standards set by the automotive industry are in place to ensure a high quality vehicle is produced, which is, ultimately, safe to drive. The use of machine vision for carrying out quality control inspections while maintaining the rate of production demanded by automotive manufacturers is a crucial aspect of the production process.
Laser welded and laser brazed seams of vehicles are one such aspect where quality control requirements must be met. The Swiss-based vision company Photonfocus provides CMOS cameras used in the Souvis 5000 weld inspection system from Precitec Vision, a member of the Precitec Group specialising in laser welding and quality inspection systems.
Vehicle doors and roofs are manufactured from a number of metal sheets known as ‘tailored blanks’, which are welded together before being formed into the constituent parts. Laser welding of tailored blanks is an important process in the automotive industry and is closely monitored to ensure high quality welds. The Souvis 5000 is a fully automated inspection system that monitors the quality of the welded seam.
The inspection process occurs shortly after welding, simultaneously analysing images for 2D and 3D weld geometry, weld texture, and any imperfections, such as pores, in the welded seam. The weld needs to be homogeneous along its length, with any variations in profile or pores in the seam weakening the joint. The system has set limits for the various criteria under analysis and will alert the operator to any sections of the weld that require reworking or need to be scrapped.
To analyse the 3D seam profile, two diode lasers mounted in the sensor head project lines of light across the surface of the joint to determine whether the seam is excessively concave, a technique termed ‘laser triangulation’. At the same time, greyscale images are taken and used to detect any pores or cracks in the seam.
One of the challenges of this system is to capture very bright profile line images and low-light greyscale images simultaneously. ‘In a scenario with strong illumination there is the possibility of the sensor becoming saturated with light,’ explains Dr Renate Petry, field application engineer at Photonfocus. The cameras used in the system utilise Linlog technology, developed by Photonfocus, which provides a linear response at low light levels and a logarithmic response at high light levels, allowing both greyscale and laser triangulation images to be captured.
An essential criterion for weld inspection, according to Petry, is the CMOS sensor, which allows flexible windowing, in that only the region of interest around the weld seam needs to be captured and analysed. This speeds up the camera and allows very fast processing. Furthermore, the high dynamic range available with CMOS sensors, which in the case of Photonfocus’ camera used in the Souvis 5000 can reach up to 120dB, avoids very bright areas dominating the scene.
Image capture and processing is extremely quick, with the weld seam undergoing inspection at a rate of 6-30 metres per minute. The inspection system has been employed by various automotive manufacturers, including Audi, BMW and Daimler.
A further example of an inspections system is that developed by Aicon 3D Systems, a Germanbased provider of 3D measurement systems. The TubeInspect is an inspection system for tubes and pipes, such as those used for fuel or brake fluid. The system is an optical gauge that compares the dimensions and shape of the pipework against CAD models detailing the exact specifications of that particular length of pipe. In this way, the manufactured pipe can be checked to ensure it has been bent into the correct shape.
Oxford Sensor Technology’s (OST) vision sensors are used in an automated glazing system, where windscreens are fitted into a vehicle body as it moves along a conveyor belt. Image courtesy of OST.
The TubeInspect system uses 16 high-resolution Marlin cameras from Allied Vision Technologies (AVT), a German machine vision company. ‘We are using the principle of photogrammetry, in that we don't measure the tube itself, but capture pictures and make measurements from those,’ explains Guenter Suilmann, head of sales and marketing at Aicon. To be able to do this, high resolution cameras, which can capture enough detail, as well as software dedicated to making the measurements, are required in a system such as TubeInspect.
The tubes are shaped into complex convoluted patterns by bending machines, the dimensions of which need to be precisely controlled. Space underneath the car is often restricted and there are also safety concerns with installing pipework that is not shaped correctly, especially in the case of critical parts, such as the brake fluid line.
Traditionally, pipework was checked using mechanical gauges, which is a costly process, as gauges had to be made for each tube and vehicle type and calibrated accordingly. In addition, any changes in pipe geometry meant adjusting the gauges. The TubeInspect system provides an alternative measurement device. The 16 cameras are synchronised to provide complete coverage of the tubing, capturing images from the entire length in fewer than 3ms. The analysis of the image, including the comparison against the CAD model, takes approximately 10 seconds, with a measurement accuracy of down to +/-0.025mm in the latest version. ‘You are able to look in detail at any part of the tube that is not acceptable and send the correction data back to the production machine,’ explains Suilmann.
Suilmann expects more and more automotive parts to undergo a similar inspection process as that carried out by the TubeInspect. Currently, not all parts are inspected in this manner, as there isn't the requirement to do so. However, with inspection systems carrying out increasingly rapid appraisals of the parts, he believes that more optical inspection will take place in the future.
Quality control inspection is also important for checking the various components that make up the completed vehicle. Industrial Vision Systems (IVS), an Oxfordshire-based manufacturer of machine vision systems for various industries including automotive, has been working with a UK-based OEM manufacturer of automotive door handles to develop a vision system for inspecting its products.
An automotive handle is made up of a number of constituent parts including the main body of the handle, a buffer insert and a clip, all of which are critical to the effective operation of the handle. 60,000 door handles are produced by the manufacturer each day and quality control is crucial.
IVS’ inspection system was set up to carry out quality control checks on handles being fed into the machine on a conveyor belt. A barcode, read via the vision system, gives the settings of the current handle type, which ensures the correct check routine for the specific parts is carried out by the vision system.
Checks made include: that the correct combination of parts is present in terms of left hand/right hand handles and locking/non locking parts; that the individual components are seated correctly; and that the positioning of the parts within the customer’s box before final delivery is accurate.
Six cameras mounted into the canopy over the conveyor carry out the checks – five medium resolution digital FireWire cameras and a further high resolution camera. The medium resolution cameras are used for general inspection processes, including the overall correct presence of the handle and deviations of sub components within the handles, while the high resolution camera is used for label inspection and reading the barcode.
The vision inspection area is purposely designed to stop the ingress of ambient light to control lighting levels. The lighting units, made up of high intensity 280mm white LED line lights, include a polarising filter to reduce glare and reflections from the metallic painted parts.
Precitec Vision’s Souvis 5000 weld inspection system uses cameras from Photonfocus to inspect laser welded and laser brazed seams on vehicle body parts. Image courtesy of Precitec Group.
The whole vision system is controlled by a standard NeuroCheck machine vision software package, which communicates with the cameras. The software carries out any image processing to confirm the acceptance or rejection of the part. To supply the system’s operator with information regarding each box inspected, IVS uses the NeuroCheck software to develop a graphical user interface to display images and the results of each inspection. As each box passes through the system, the operator is presented with an image of the box and the highlighted search regions. The results of each pass or fail on each box is displayed, as well as a running total of the number of handles inspected and passed by the software.
Apart from quality control inspection, machine vision is utilised heavily within the production process itself, whether that be identifying and sorting components or aiding robots in pick-and-place applications, in which vision systems guide robots in locating items on a pallet and placing them on a production line.
The information to ensure the correct tyre is fitted onto the correct wheel is supplied in DOT codes, alphanumeric sequences used by the US Department of Transportation for identification of tyres. The code is moulded onto the tyre and contains information on tyre type, and the location and date of manufacture. DOT codes are a legal requirement in the US, and with modern cars using an increasingly complex array of tyre variations – Jaguar produces a vehicle model with different tyres on all four wheels – accurately reading codes is important.
Oxford Sensor Technology (OST), a UK-based supplier of machine vision systems for robot guidance, inspection and process control, has developed an automated 3D scanning system to read DOT codes. The system utilises an 80mW laser diode to project a line of light, 35mm wide, across the tyre wall, and a Sick IVP Ranger camera to view the laser line at an angle. Undulations in the tyre surface are denoted by alterations in the laser line, which is subsequently recorded by the camera. The tyre is rotated and the camera records where the laser line falls on each column of its detector, rather than capturing complete images, at a maximum rate of 30,000 times a second. In this way a 3D model of the tyre surface is built up.
OST’s system operates at 667 images per second, with the resultant image measuring 1.4m long by 35mm wide, and captured in six seconds. The 3D model is converted into a greyscale image, with the brighter parts of the image representing raised parts of the tyre. Text recognition software is then used to identify and read the DOT code.
Manual inspection of DOT codes is labour-intensive and prone to errors. In automating the process, a vision system had to be developed that could read black text on a black background, something which is incredibly difficult to achieve using a static camera system. Anthony Williams, managing director of OST, explains: ‘The human eye is very clever and can automatically adjust to lighting conditions to focus in on certain areas. You can also move the object in order to view specific details,’ he says. These attributes aren’t available with a static camera system.
Industrial Vision Systems’ (IVS) inspection device carries out quality control checks on automotive door handles. Image courtesy of IVS.
Williams suggests that to obtain this information using a static system, an exceptionally high resolution camera would be needed to take a picture of the entire wheel and, even then, it is not easy for imaging systems to read. The DOT code is also difficult to identify on a tyre using a static image, as it is positioned differently depending on the tyre. With OST’s system, the tyre is rotated and the DOT code can be identified easily.
OST’s vision sensors are also used in an automated glazing system, where windscreens can be fitted into a vehicle body as it moves along a conveyor belt. The system uses four SRS VisionSensors mounted onto a robotic arm to detect the specular reflection given off by changes in the profile of the car body.
Traditional vision systems rely on the scattering of light from the vehicle’s paint to allow the sensor to guide the placing of the glass. The light reflected differs in brightness depending on the paint colour, with a white car reflecting light up to 1,000 times brighter than that reflected from a black car using the same light source. As a result, traditional vision systems don’t operate over the entire range of colours car manufacturers produce and some windscreens need to be fitted manually. The SRS VisionSensor overcomes this by measuring the specular reflection on the rounded surfaces of the vehicle’s body, which are constant irrespective of colour.
OST has installed SRS VisionSensor systems in production plants of Ferrari, Renault, Ford, Jaguar, Land Rover, and Valmet (Porsche), to name a few.
Williams states that ‘build quality is becoming more of an issue [in the automotive industry]’. Components, such as the dashboard, are inputted using vision-guided systems, which are important in ensuring the best possible fit. In addition, inspection systems ensure the build quality of the vehicle meets the high standards of manufacturers.
He also points out that car manufacturers are looking to automate more and more processes that traditionally were, and still are, carried out manually. One such example Williams cites is an automated refuelling system that OST is currently developing. At present, a certain amount of fuel is manually put into the car to drive it to the showroom. It is hoped that the vision-guided system will cut labour costs, increase productivity, and improve health and safety for employees, who will no longer be required to carry out this task.
Luxury vehicles are equipped with all sorts of gadgets and technology to maximise driver comfort. Kongsberg Automotive, a Norwegian manufacturer of systems for gearshift, clutch actuation, and seat comfort, designs and manufactures cloth containing electrical wiring and circuitry for seat heating in cars. Machine vision is involved in quality control inspections, ensuring that wires are correctly positioned to allow enough margin through critical cutting zones and that adhesive is properly applied. Scorpion Vision, the UK distributor of Scorpion Vision software from Tordivel, a vision company based in Oslo, Norway, supplies its software for use in the quality control checks.
Processing the material involves backlighting the cloth on a ‘line up’ table, which contains a transparent template with guidelines showing the correct position of wiring and adhesive. ‘The cloth works as a lowpass filter,’ explains Paul Wilson, managing director of Scorpion Vision, ‘and we must be able to detect the guiding lines on the transparent template sitting under the cloth.’ Scorpion Vision’s LineFinder tool has the ability to detect lines in very low contrast images, such as those taken through cloth.
Test areas are set up using ToolBox and Python scripting support, in which the wire is detected and the distance to cutting zones is calculated. The resampling tool is also utilised to produce images with no lens distortion, as Wilson explains: ‘The usage of wide angle lenses is necessary to cover a full sheet without having to haul the camera close to the ceiling. The downside of this is pillowshaped images that may confuse an operator and challenge the engineer when working with the [software's] tools.’ The resampling tool allows engineers to work on an image free from lens distortion.
Scorpion Vision software is used to ensure electrical wiring and circuitry built into car seat material is intact and positioned correctly. Image courtesy of Scorpion Vision.
The Scorpion Vision software is also used in a robot guiding system, where, as the material arrives at the robot station images are relayed back to the control unit and the position of wiring is computed and coordinates are sent to the robot. The automated system then applies adhesive in the correct position.
End-of-line inspection is carried out as well, with the product being identified by barcode and the corresponding template and set of analysis tools loaded onto the software. Checks for broken or unaligned wiring, or misplaced adhesive, among others, are made.
Scorpion Vision software can handle many inspections and many product variants, says Wilson, which is important in the automotive industry where quality control checks are numerous. One of the features of the software that Wilson cites is the linking of tools in a coordinate system through reference points. ‘The Python scripting support makes it possible to drive a collection of tools from a script, and by just changing the initial reference you may apply the collection of tools to similar structures elsewhere on the product.
‘Kongsberg Automotive products often contain material with two or three products per sheet. By creating a solution for the first product, we can apply the solution to all other similar products on the sheet,’ he says.
Wilson expects future developments in the use of 3D technology to control robotic systems: ‘By using stereoimaging and 3D techniques, vision systems are able to resolve the Z dimension as well as the X and Y, providing robot applications with even better support.’
