Machine vision is associated with the prescriptive environment of a production line, where the lighting conditions, environment and the objects under inspection are predictable and consistent. In heavier industries such as mining, however, these factors can be far less regular, and the scale of vision systems required can bring challenges of its own. This feature focuses on the use of imaging technology in three large-scale industrial applications – open cast mining, railway maintenance, and storage of hazardous materials. While the imaging systems used in these applications differ greatly from one another in terms of both function and technology, the scale of these heavy industrial applications leads to some common challenges.
Trains on time
Safe rail travel depends on proper maintenance of the railways, with drastic and often fatal accidents occurring where rails are allowed to deteriorate beyond a predefined wear limit. Maintenance work on railways is done at night (or sometimes at weekends), and has traditionally required a highly trained inspector to walk along the track, checking the rails first-hand for damage and metal fatigue.
Chris Baldwin, business development manager for rail markets at Stemmer Imaging, explains that requiring inspectors to walk down the rail line is both difficult and dangerous: ‘The operator has to close the train line before sending people down on foot, and the staff overheads required are huge,’ he says, explaining that one person walking down a rail requires approximately four other people working away from the track to ensure his or her safety. ‘They have to take particular steps to make the rail system safe,’ he adds. ‘For example, percussion caps, which have to be bolted onto the rails hundreds of metres away from where the workers are in order to warn of any approaching trains, and also to give any train drivers a safe stopping distance.’ In-situ human inspection is, therefore, a significant expense for rail network operators.
Stemmer Imaging is working closely with a number of rail networks to develop automated systems that eliminate the dangers and expense of human inspection. ‘The concept is to be able to run a specially equipped train down the track, at full speed, and to gather all of the information you need to be able to do full safety analysis of that track – making sure that everything is in place, and that everything is working correctly,’ says Baldwin.
The driving force behind this automation is more than cost savings and the safety of the inspection team, and extends also to the repeatability of the inspection process, and the ease with which it can be recorded. ‘Humans are very good at the task of inspection, but they’re not 100 per cent accurate,’ explains Baldwin. ‘If a track inspector is tired or hungover one day, he’s going to do a different job to the days on which he’s not tired or hung-over.
Similarly, on an overcast day, an inspector will probably be able to see more than on a bright sunny day or on a wet day. We’re trying to remove the human variance in gathering the data, before presenting that information to a human inspector. A human is still the safety interlock in all this, inspecting the imagery to make the decision of which particular rails need attention due to wear, for example. These inspectors have been trained for a long time in order to be able to do their job, but instead of taking the expert to the track, we take the track to the expert.’
By logging inspection data digitally, rail networks have more opportunity to optimise maintenance. ‘Maintenance of railway tracks is continuous and ongoing, and sometime a single piece has to be replaced more often than others. Instead of going up to the track and saying “this rail is damaged and must be replaced,” they can actually monitor the process of wear and predict maintenance requirements a long way in advance,’ he adds. ‘They can say “this piece of track doesn’t need replacing now, but judging by how it was wearing last year, it will need to be replaced soon.” If a certain part of track is always wearing out, perhaps it should be changed for a more durable type of rail.’
Thermal imaging has been used to spot hotspots in coal stockpiles, preventing spontaneous fires from breaking out.
Because inspectors are used to performing this analysis using their own eye, the cameras sold by Stemmer for these applications must be high resolution and high fidelity, so that the images are not only repeatable, but also pleasing to the human eye and representative of nature. ‘Normal machine vision usually uses monochrome images, but humans take a lot of information from colour, such as rust for example,’ says Baldwin. ‘Certain parts of a rail should be shiny, while others can be expected to be rusty, and so colour is useful.’
In practice, high-resolution monochrome IR cameras are installed underneath the inspection train, and colour data is provided by a colour camera on the front. ‘We need to be able to tell the system to take an image exactly at a specific point in time, and that specific point is then mapped using a piece of location hardware on the train,’ says Baldwin. ‘We have to meet very strict health and safety rules whenever we’re working with trains. The primary consideration is that if we bolt something to a train, it mustn’t hit anything and it mustn’t fall off.’
These health and safety considerations, he explains, are the most significant decider of the kinds of technology that can be used. IR light, for example, is used underneath the train, because there are limits to how bright a visible light source could be in a non-standard part of the train. Additionally, the IR source must be LED-based and bellow 70°C at its hottest so as to prevent fire. The IR LEDs are strobed at a high intensity, synched with the camera shutter, and the duty cycle is kept low. The light source for the colour imaging on the front of the train, on the other hand, can be a standard high-brightness discharge lamp, as trains are expected to have bright headlights.
GigE Vision cameras are used, he explains, as the distances between the cameras and data storage units can be up to 40m. Additionally, the signals are converted and transmitted fibre optically, which Baldwin notes is a cost effective way of transmitting a signal in an electronically noisy environment (high voltage power lines and static).
Currently, each network will generally equip a single, specialist train with the cameras and equipment necessary to inspect a line, but Baldwin sees this changing in the future. Camera systems will one day be fitted underneath ordinary passenger trains to allow maintenance inspection at any time, day or night, thus minimising the need for ‘planned engineering works’ dreaded by so many weekend passengers.
Keeping cool
Thermal imaging giant Flir is usually associated with military and security applications rather than heavy industry, but the company has found a lucrative and high-volume use for its technology in coal mining, as well as in other sectors where flammable goods must be stored. Coal stored in large stockpiles brings with it a risk of spontaneous fires, as Chris Brown, European business development manager for automation at the company, explains: ‘It’s a pretty regular occurrence. Gasses build up within the coal pile, and this can start to generate heat.’
Varieties of cameras are fitted to line inspection trains in order to safely and quickly monitor the wear of sections of track.
Enelex, a Czech integrator, has used Flir’s radiometric thermal imaging cameras to create automated systems for predicting and preventing these fires before they break out in the coal yards of the Nástup Mines Cooperation in Tusimice, Czech Republic. A fully-automated system is required in order to monitor the thousands of tonnes of coal stored on the 200 x 800m site. The system is based around radiometric thermal imaging cameras. ‘Not only do these display a thermal image, they also allow the users to measure temperature,’ explains Brown. The cameras are able to measure temperatures between 20 and 2,000°C to an accuracy of two per cent or 50mK.
‘In these applications we can point the cameras at the coal pile, wood pile, or whatever else we’re monitoring, and then we can specify an alarm temperature range. If, for instance, the camera sees anything in the area that’s above 50°C, it is told to take a picture of it, email it to somebody, and raise an alarm via the I/O on the back of the camera. The camera is acting intelligently, and is able to take a number of actions when it sees a fire risk.’
Compared to the image processing features of some modern smart cameras, Brown concedes, the abilities of these thermal cameras are relatively simple. Fire prevention, he says, is a massive market, and these cameras are well-adapted to serve their niche well. He notes that the company sells around 40 such cameras each month, just within Europe.
Monster trucks
Also in the mining sector, Matrix Vision has provided a 3D camera system for a sophisticated machine vision implementation that aims to prevent pedestrians from ending up on the wrong side of a monster truck. The dump trucks used in the mines in Africa and Australia, for example, are among the largest vehicles in the world, with some able to carry more than 100 tonnes of rock. Unsurprisingly, the visibility from the driver’s seat is minimal. Reversing sirens are mandatory in many countries, but the efficacy of these auditory warnings is nullified by an iPod or telephone, and accidents do happen.
Integrator Arcure (Paris, France) developed its Blaxtair system consisting of externally mounted pairs of stereoscopic cameras that monitor the 3D environment of the truck, tracking all objects within it. The system is capable of recognising the shape of a pedestrian regardless of his or her position (e.g. standing or kneeling) and will warn the driver in the event of a safety risk. The computations are carried out on an FPGA within the cameras at a refresh rate of 10Hz, meaning that the driver will be alerted to any obstacle within 300ms of its detection.
During development, Arcure worked closely with Matrix Vision to find a camera suitable for the system, as Ulli Lansche, technical editor at Matrix Vision, explains: ‘Arcure made long tests of several digital cameras from different manufacturers and finally chose the mvBlueCouger-X from Matrix Vision.’ The feature that made this camera suitable was the highly accurate synchronisation between master and slave cameras, achieved using timestamps, which was vital to Arcure’s proprietary stereoscopic system. ‘Arcure developed the system, but we supported them with upgrades. They said what they needed in the hardware – better synchronisation between the two cameras – and we upgraded to be able to supply it in our system. Now all of our customers can use these features,’ says Lansche.
The harsh environment of mining placed additional constraints on the selection of cameras for this application. On a sunny day, temperatures in the camera enclosures can reach 70°C, which the camera can tolerate; its low power consumption, Lansche adds, prevents the temperature from rising further. As was the case in the train application, GigE Vision was the communication protocol of choice because of the distances involved in larger machines – up to 20m on some of the true giants.
Bigger and better
The industrial applications described here have had safety at their core, but as imaging technology becomes more integrated into industry, perhaps one day the human element will be removed entirely. Will enormous machine vision-guided machines automatically navigate the mines of the future?
