A driverless vehicle at the Darpa Grand Challenge 2005
Cameras are clearly having a much greater role in transport; driverless cars at the Darpa Grand Challenge 2005, guided solely by machine vision, travelled a 212.4km course across rough terrain. Of more practical use, BMW has designed a system that guides cars in a straight line on the motorway, and another system wakes up the driver when it sees his or her eyelids are drooping.
The Ladybug from Point Grey Research, which provided some of the camera systems for the Darpa Grand Challenge, is used in virtual mapping exercises by various states in the US. The system works in conjunction with a GPS system, and contains six cameras – five in a circle, and one on top – that simultaneously capture images covering 75 per cent of the sphere. The images are used in inventory management systems of roads, cataloguing every sign.
However, by far the most common application of imaging in transport is licence plate recognition, used to catch speeding vehicles, and in traffic light adherence. For this kind of application, high-resolution images, controlled using a trigger mechanism, are necessary. In the US, colour images are needed to identify the vehicle properly, but this is not the case in all countries. The cameras are controlled by a computer, so a FireWire interface is often used.
Tolling applications require a more sophisticated system. Sandy Keen, from Point Grey’s marketing department, says: ‘It’s phenomenal how these systems work.’ A radio transmitter is fitted to the car that indicates whether the driver has paid. If the system recognises that the car has not been paid for, it captures an image of the vehicle and records its registration number.
Obviously, accuracy is very important in these applications. The cameras need to have a large optic dynamic range, to cope with the significant variance in intensity caused by bright reflections. Cypress Semiconductor provides CMOS sensors with a multiple slope function that varies the integration time from pixel to pixel depending on the intensity, making the picture much more uniform. Tim Baeyens, director of marketing and sales at Cypress, describes why CMOS sensors are most suitable for this kind of job. ‘Unlike a CCD sensor, the CMOS sensor does not suffer from blooming. This is where a spot of very bright light causes the electrons on one picture to spill over onto neighbouring pixels, causing a star-like stripe. This can even affect the next few images.’
Clearly effective illumination is an important feature of the systems. PerkinElmer provides stroboscopic lighting for licence plate recognition systems. ‘Some people are trying to use LEDs, but they can’t illuminate from such a large distance,’ says Kenneth Reid, an applications engineer at PerkinElmer. ‘They are more of an industrial technology.’ PerkinElmer’s light source is a second-generation, pulse Xenon lamp that is much brighter than its competitors and has been used for traffic surveillance systems in Italy for three years now. ‘We’re trying to be very flexible in the configuration we’re offering. The power supply can be far away from the flash lamp, and we’re offering to lay the cameras at 90° to one another to get pictures at different angles, rather than just from the front and back.’
Software that can capture and analyse the images completes the process of licence plate recognition and traffic analysis. Abstract Computing has recently joined Sony’s Vision Network, so that its Universal Licence Plate Recognition software will be integrated onto the processing chips of Sony XCI-SX1 and XCI-XV3 smart cameras, opening up mobile applications. Currently, Abstract Computing systems use wireless, GPRS communication to allow even greater mobility.
The collaboration started three years ago, and the corresponding products have already been bought and used by a number of companies. It is common for vision systems to be used to collect data about traffic infrastructures; The Dutch government recently used this software to measure the effect of dynamic road signing, and it is feasible that vision systems could be used in the end product too. The systems could be used to measure the number, size and speed of vehicles on the road. Allard Blom, chief operational officer at Abstract Computing, explains the advantages of using an imaging system: ‘The information is much richer. In the past, an inductive loop laid across the road was used to record this information, but it had a higher cost and was less accurate.’
Obviously, many people may be uneasy about the amount of information that is being recorded without their knowledge. However, there is no need to worry, according to Allard. ‘We have created a specialised system that encrypts the license plate so you can only tell the information about the vehicle’s movements, and not about the actual car or its passengers.’
Independent companies are also using the software for parking and security purposes. It can record the number of entries and exits, and report on suspicious vehicles. In some implementations, the registration number has been linked to an email address, to notify businessmen when a client is about to visit.
Abstract Computing produces another piece of software, called Universal Motion Software, that is also used in traffic analysis. ProRail, the Dutch rail service, however, found a more unusual application. They deployed the software to analyse the movement of travellers within train stations, so they could better design the interior of the stations. The software counted the number of people per square metre at the busiest times of day to find the most congested areas.
Video analysis is equally important, and can make different applications possible. AnaLogic Computers has recently added traffic monitoring to its security/surveillance analytics capabilities for embedded systems. The complex algorithms can process images from traffic cameras to identify a wide range of behaviours, such as lane changes, unauthorised vehicles travelling in bus and emergency lanes, and other forms of irregular driving.
Firstsight Vision has recently announced that Manto, its pattern-recognition tool, has also been used in pilot applications for traffic analysis applications. Manto was originally used to classify organic objects, and even has the capability to identify gender differences in human faces. It would classify the cars either by type of vehicle, or by manufacturer and model.
Steve Hearn, sales manager at Firstsight Vision, says: ‘It is very reliable, and could be used instead of a human to make a decision. Rather than having five people sat at a junction counting the number of vehicles going past, a camera could record the results automatically, without losing concentration after a few hours.’
The software creates classification criteria, called a decision surface, based on observations of correlation, geometrical connections, texture and colour. The decision surface is generated by a series of training images, and its results improve with experience. The neural technology that powers this, called Support Vector Machine, has the ability to recognise objects that it has not seen before. After each categorisation, a confidence factor is generated; if this is below a certain threshold, the image is stored and an administrator will make the correct decision, providing further training for the system. Only the decision surface, and not the image database, is stored, making it impossible to over-train the software, and minimising the ultimate size of the file.
Together with Firstsight Vision, Manto has been working on feasibility studies for the last 18 months. Currently, Manto is 85 per cent accurate, but it is hoped this could increase to 95 per cent when a final end product is available. ‘Because we are just a supplier, we are not close enough to the final application to finish it off,’ says Hearn. ‘We’re looking for partners to put it in a wrapper to match what the end user is looking for.’ He hopes that Manto will be used in traffic flow monitoring, surveying, and congestion charging. It could also be used in intelligent search algorithms for security applications. For CCTV, each vehicle could be labelled, making searches for a particular vehicle much quicker than a human scrawling through the actual footage.
With so many applications planned for the near future, and new technology enabling more efficient, mobile implementations, there seems little chance the growth of this market is going to slow down.
