Walking into the circular glass building of Factory 2050, part of the University of Sheffield’s Advanced Manufacturing Research Centre, you get a sense of what goes on at the facility from the jet engine on display in the entrance hall.
Rolls-Royce, Boeing, BAE, Airbus, GKN Aerospace, Safran, and Spirit Aerosystems have all invested in the site, built on an old Sheffield airfield, where engineers now work on new manufacturing methods that could produce the next generation of aircraft. Boeing’s first European factory is at the end of where the runway used to be, and Rolls-Royce and now McLaren, the AMRC’s newest member, have facilities here.
Kieran Edge, AMRC’s technical lead for machine vision at the Integrated Manufacturing Group, showed me some of the work at Factory 2050. This includes an automated inspection cell funded by Rolls-Royce, a digital assembly cell that shows how augmented reality and collaborative robots can aid manufacturing, alongside many other manufacturing and machining work. It’s not just aerospace projects; the AMRC also works in construction, medical, automotive and energy sectors. One of the demonstrators Edge showed me was a rig inspecting flat pack houses. It uses a pan-tilt-zoom camera and machine vision software to identify holes for light switches, wiring channels, and other features. A 3D surface analyser is also positioned above the panel to check for warping in the wood.
The direction of AMRC’s research is driven primarily by its members, of which there are two tiers. The first tier of companies, which includes Boeing, Airbus, BAE, Rolls-Royce, McLaren and Siemens, contribute £200,000 a year in membership, although that is subject to change. They have a seat on the board and have input into the direction of research. These firms suggest projects, the IP from which is then shared among members so all can benefit.
The second tier of membership pay £30,000 a year, for access to the shop floor and the general research at the AMRC. Multipix Imaging is a tier two member; it provides 2D and 3D cameras, along with MVTec Halcon software licences. Kuka is another tier two firm. The centre boasts the world’s largest high-precision machining robot, the Kuka Titan, which was designed originally to pick up cars in automotive factories, but which the AMRC is using to machine aluminium and titanium.
The centre also gets funding from the UK government, from high-value manufacturing Catapult funding – the digital assembly demonstration cell that uses augmented reality and collaborative robots is funded in this way.
The final investment stream comes directly from customers. The Rolls-Royce automated inspection cell, for example, is a specific project paid for solely by Rolls-Royce, for which, when it’s complete, Rolls-Royce will own the IP. This four-year project is scheduled to finish in 2020.
The centre also runs short assist projects with small companies. It has worked with a hand tool manufacturer based in Sheffield, called Footprint Tools, to automate some of its production processes. It’s also worked with the Cardboard Box Company, in Accrington, UK, to show how to palletise boxes from a conveyor belt using vision technology.
‘Our base is in aerospace, but we’re moving towards automotive, rail and construction,’ Edge told me. ‘A lot of the technologies we use in aerospace projects are transferable and applicable to other industries.’
The Integrated Manufacturing Group is based at Factory 2050 on the Sheffield business park. Nearby are the Machining Group, Castings, Design and Prototyping Group, Composites Centre, Structural Testing Centre and Medical AMRC, which are all based on the Advanced Manufacturing Park (AMP) in Catcliffe, Rotherham. The AMP is also home to the AMRC’s sister centre, the Nuclear AMRC.
Artificial intelligence
One of the demonstrators Edge showed me was a robot trained to understand its movements using artificial intelligence. The system displays a circle in a random location on the floor. The robot then tries to place a puck in the centre of the circle, and a camera tells the system the alignment of the puck and how close it gets to the centre. The camera is feeding in data about where to position the puck, then seeing how accurately the robot can follow those commands. The robot has been running for several weeks now, gathering data points and learning along the way.
‘Robots are very repeatable but not very accurate – they’ll go to the exact same wrong position every time,’ Edge said. ‘We’re trying to train the robot to move more accurately.’
The AMRC has four projects looking directly at artificial intelligence for machine vision inspection, including a smart factory platform that’s using a neural network to identify faults during part assembly. A Basler camera, supplied by Multipix Imaging, monitors the assembly process, while Halcon software analyses the images. The centre has an entire AI theme, investigating natural language processing as well. It’s using IBM’s AI platform, and also assessing AI software from Microsoft, Cognex and MVTec.
Edge said that aerospace firms, like many companies, are interested in exploring the capabilities of AI, but that validating the technology would be a big task. ‘Validating machine vision alone is difficult,’ he said, ‘and you know what the algorithms are looking for, and how it will fail.’ An edge detection algorithm, for example, will fail to identify a defect if it doesn’t have a strong enough edge. However, with AI, Edge continued, the neural network has been trained on the data, but the user doesn’t quite know what features the network is looking at, and whether or not it will fail.
The AMRC's demonstration lathe on which it has installed numerous sensors to show how well the machine is running
This makes gaining confidence in AI a difficult task, especially in aerospace, where the sensitivity of inspection is paramount – missing a defect on an aerospace component could be very costly.
‘It’s not just about getting the data but getting the right data,’ Edge said. If a defect appears once in every 1,000 components, it’ll take years to reach a sufficient level of data to have confidence in the technology. ‘It’s a business case on whether you bite the bullet early and start doing this data collection,’ he continued. ‘It is an area where companies like Rolls-Royce are starting to look forward in their data gathering and annotation, and having these stockpiles of data.’
The AMRC is starting to investigate how best the data from the Rolls-Royce automated inspection project can be used, including whether it can be used for AI. The cell generates a large amount of data per component, according to Edge. ‘It’s a lot of data, especially when they’re making 100,000 of these components a year,’ he said. ‘It’s what you do with that data, how you label it, how you annotate it, and how you store it.’
AMRC is working to assess the risk of a lot of these technologies. It will put them on the shop floor and simulate a production environment, using real production parts. It also tries to shorten the period of adoption for some of this technology, and limit the time between a concept identified by a university and technology that can be put on the shop floor.
‘Machine vision can be considered a risky technology,’ commented Edge. There are challenges involved in acquiring data, especially data about defects. Understanding how good the system is, relative to a human inspector, is also very difficult, because it’s hard to assess the capabilities of a human inspector without bias. The main risk, though, with using machine vision, is that if a machine misses a defect, then it’s reasonable to think the machine would always miss the defect and that it’s missed those defects in the past, because it’s the same algorithm. If a human misses a defect that’s human error, which is more of a one-off event, and the operator can be re-trained. But if a machine misses a defect, that can be a repeatable, mass-recall error. ‘That is a big hindrance for machine vision systems,’ Edge said. ‘Even if the system is better than a human, there’s trust in the human operator.’
Factory 2050 is the first building to be completed on the University of Sheffield's advanced manufacturing campus
The AMRC has been using AI for object detection and scene classification so far. Artificial intelligence gives better classification results than traditional machine vision techniques, according to Edge. Detecting scratches, for example, is where AI excels, but validating the technology will be difficult. Edge suggested two possible ways of validation: leaving an AI system running blind for a year or so, and comparing the results to reports from a human operator. Or, use two AI systems, one looking for specific defect types, and another running in the background looking for anomalies and things it’s never seen before, which would act as a failsafe. It’s then up to the customer whether it can swallow the initial period of gathering data, in order to benefit further down the line.
‘There are a lot of different aspects in aerospace that want to [use AI and] jump on this bus,’ Edge said. ‘But the first person who jumps on the bus has to buy the bus. That’s one of the barriers to entry.’
Connected factories
‘All this digital data is going to become more prevalent moving into the next generation of aircraft,’ Edge said. Aerospace factories are investing in Industry 4.0 and connected manufacturing, just like many other industries.
This applies even to older machinery. The AMRC has a demonstration lathe on which it has installed various sensors to measure carriage position, spindle vibration, motor current temperature, and other variables, all of which is collected on a Raspberry Pi. All of AMRC’s partners have lathes and machinery like this still in operation in their factories, and they don’t want to change that, Edge said. The data can be used to tell the shop floor manager which machines are running, which are idle, along with uptime and cost efficiency. The sensors also give an idea about how well the machine is running, allowing engineers to carry out maintenance before it breaks down. The sensor system was put together for a few hundred pounds worth of hardware, Edge explained, built in an open-source network GUI.
Another smaller, three-month project, currently underway for Rolls-Royce, is designed to show how machine vision can be used to distinguish between components. The parts are very similar in appearance, apart from one or two features like the placement of a bracket or the number of holes, which makes them difficult to tell apart by a human operator. ‘We can say what the barrier to entry is,’ explained Edge. ‘It’s up to Rolls-Royce to decide if there’s a business case for the technology.’ In this case, AMRC is using a 30-megapixel camera supplied by Multipix Imaging, which is over-specified but useful for this proof-of-concept work. ‘This is an area where our relationship with Multipix has helped us out,’ Edge said. ‘It allowed us to do these types of operation pretty quickly.’
Another type of technology that AMRC is working with extensively is virtual and augmented reality, showing how commercial headsets can be used for plant simulation, assembly, training instructions and other functions to save costs. In the construction industry, for example, AMRC has converted electrical wiring instructions into an AR overlay to highlight which terminals should be connected to each other. The technology prevents mistakes, Edge said, and is far more cost effective than having to train an engineer to wire different types of cabinet – instead, upload the right cabinet make and the engineer follows the AR overlay instructions.
‘Overall, machine vision is of massive interest in UK manufacturing,’ Edge said. He feels one big issue companies often have is that machine vision is fitted retrospectively and trying to work around other areas of manufacturing. ‘All these machine vision problems would be a much easier job if you had a dedicated space to do it in,’ he said.
‘We’re still growing,’ Edge concluded. ‘Inspection is so critical in aerospace; everything is inspected, a lot of which is still manual and can be improved on.’
Working within aerospace?
Would you like to share your experience of developing and deploying an inspection system for an aerospace customer? Please get in touch: greg.blackman@europascience.com
