Monitoring the flow of traffic on London’s roads, as is the case in many cities, is still largely based on footage from CCTV cameras. Digital imaging techniques are also used and might come into play more often in the future, but existing analogue technology is still adequate for a number of traffic monitoring tasks. A project currently being undertaken by the Digital Imaging Research Centre (DIRC) at Kingston University, London, in collaboration with Transport for London, is using computer vision analysis to add more intelligence to the images from CCTV cameras. Algorithms for vehicle classification and tracking based on CCTV footage are being developed to improve traffic flow monitoring and enforcement in the capital – bus lane enforcement or detecting illegal turns, are two potential areas where the added intelligence could be beneficial.
In general, traffic monitoring applications can roughly be split between law enforcement and traffic flow control. ‘Images captured for speed enforcement cannot be manipulated in any way and therefore uncompressed data is used. This is one of the biggest differences between enforcement applications and traffic flow control,’ comments Henning Tiarks from Basler Vision Technologies.
Basler, with headquarters in Ahrensburg, Germany, produces vision solutions and components for a number of sectors, including for Intelligent Traffic Systems (ITS), which Tiarks comments, is one of the areas outside of the industrial sector where Basler has seen high growth. Tiarks explains that traffic flow control is a fairly basic application, using simple software packages to flag up any incidents in the scene. A typical camera for these applications is a basic IP camera with ethernet connectivity and speed, one- to two-megapixel resolution, and image compression, which allows use in wide area networks (WAN) without filling up bandwidth.
The cameras used for enforcement and also tolling, which Basler classifies in the same category, are generally higher-end equipment than those for traffic flow control. ‘Here, system integrators think in lanes and pixels per lane,’ says Tiarks. ‘They will determine the number of pixels required to classify a vehicle or read a number plate over a lane of traffic (typically about 800 pixels per lane) and then choose camera technology based on this and the number of lanes.’ For a two-lane highway, Tiarks suggests a two-megapixel camera is usually suitable – whereas, for a three-lane highway, he says a five-megapixel camera is often used.
Parking enforcement
Along with speed enforcement, vision is making enforcement of street parking more effective. An automated vehicle recognition system developed by Tannery Creek Systems, based in Ontario, Canada, called autoChalk, has been deployed in a number of cities in the US and Canada to enforce parking restrictions. The system is used to image and recognise vehicles digitally by shape, size and colour, and Bill Franklin, president of Tannery Creek Systems, comments that ‘it is about five times more productive than someone walking the beat’.
‘There are a lot of cities in North America that don’t want to charge for downtown parking, but do want people to leave after an allotted time to free the space,’ remarks Franklin. The consumer culture in the US and Canada is such that consumers tend to drive to a particular store and park nearby and so a lot of small businesses rely on adequate parking facilities for their customers. ‘These small businesses are the life blood of a city, especially in times of recession,’ Franklin says, and while parking is not the only factor to consider in maintaining a healthy business community, it is one of a number of ‘housekeeping’ issues such as good security, good lighting, clean streets, and so on, that make it attractive for people to travel to downtown shopping areas.
The autoChalk system essentially replaces someone manually checking how long a vehicle has parked with a drive-by automated system. It uses four Sony colour cameras mounted on a vehicle to capture the profile and rear images of the car, including the number plate. Checks can be made at 30 to 50km/h, with the system processing about two to three cars per second. AutoChalk then compares the physical characteristics of the car with previous information to determine, firstly, whether the system has registered the same vehicle in the same bay – and, secondly, whether that vehicle has exceeded its allotted time. If the car has exceeded the time limit, the system alerts the driver, who can then stop and issue a ticket, or, in some cities, tickets will be mailed to the car owner’s address at a later date.
The system uses the Matrox Imaging Library (MIL) as an engine for automatic number plate recognition (ANPR). The feature is built with MIL’s String Reader character recognition tool that provides a robust software solution for ANPR, processing that can be heavily hampered by lighting and environmental conditions, as well as the condition of the plate, all of which can make it difficult to get an accurate read. The system also includes various adaptations for differences in the angle of the number plate, different fonts, different lighting conditions, etc. – all of which have to be accounted for. Lighting, for instance, is completely uncontrolled – which is, Franklin points out, one of the challenges for an imaging system operating outdoors.
AutoChalk includes a GPS system that’s accurate to within a radius of a metre, to pinpoint where the car is parked, and also a scanning laser that detects the vehicle’s presence – even if the number plate is obscured. The laser also captures the vehicle’s 2D profile. As the system uses the shape, size and colour of the vehicle as well as the number plate as distinguishing features, if the number plate is blocked it still registers the vehicle. ‘Environmental factors, such as dirt or snow, can reduce the accuracy of LPR (licence plate recognition) substantially, which is why it is necessary to measure other features of the vehicle,’ says Franklin.
A further advantage of incorporating a scanning laser is that it gives an accurate measurement of the length of the vehicle (accurate to ±3cm). Franklin says that cities wanting to provide parking discounts for smaller vehicles can enforce this using the system – for instance, in Calgary, Canada, where the system has been deployed, city authorities offer a 25 per cent discount for people who park downtown with vehicles smaller than 3.8 metres in length, while other cities have approached Tannery Creek Systems for a solution to charge extra for vehicles over five metres.
Difficult conditions
The changing conditions encountered when imaging out of doors is a one of the main challenges for a vision solution for the transport sector, as Marcus Bleise, international sales manager at vision company, Matrix Vision, makes clear. In comparison with the controlled environment of an industrial setting, ‘the clear opposite is the road,’ he says. ‘Day, night, clouds, fog, rain, dust, reflections, you name it; everything influences the picture acquisition and subsequently the content of the picture.’
Bleise says that the art of engineering an effective traffic solution is to take all of these environmental factors and to exclude them as far as possible. For example, using an infrared lamp to illuminate the scene and a daylight cut filter will almost solve the changing lighting conditions, not considering the infrared from the sun and the imperfect filter curve. Auto-gain and auto-exposure functions, as used in Matrix Vision’s mvBlueLYNX-M7 camera, Bleise says, do the rest.
The mvBlueLYNX-M7 has the ability to connect two sensor heads: it can be configured with two equal sensors each responsible for one lane, for instance, or with two different kinds of sensor for only one lane, one for daylight and the other with increased infrared sensitivity (or even an infrared sensor) for night.
Stemmer Imaging has developed a software package based on Manto, part of its Common Vision Blox software, specifi cally targeted at traffic applications.
In terms of lighting, David Richards, international business development manager at Gardasoft Vision, comments that ANPR applications vary a great deal from low-speed, close range image acquisition to high-speed applications where the light source is much further away from the subject. ‘The further the working distance, the more light is required,’ he says. Gardasoft Vision produces the VTR2 strobe LED light for intelligent traffic solutions, available in 740nm, 850nm and 940nm for non invasive applications and also in a white light version for capturing colour images. The system includes rapid strobing capability for applications such as red light violation, in which, Richards notes, a series of very quick pulses are typically required.
Beyond ANPR
Automatic number plate recognition is now used widely within the traffic sector and Mark Williamson, sales and marketing director at Stemmer Imaging feels that ‘ANPR is useful in itself, but the key thing is to tie it to the vehicle through make and model verification’. Stemmer Imaging has developed a software package based on Manto, part of its Common Vision Blox software, specifically targeted at traffic applications. The module will be pre-trained for a number of traffic applications, including identification of vehicle type (car, bus, lorry, etc.), suitable for bus lane enforcement and border control. Stemmer is also conducting trials for including identification of vehicle make and model within the traffic software module. This adds a further level of identification capability for border control, in which the system can be used to ensure the number plate matches with the vehicle make and model. The software package also includes through-the-lens vehicle triggering capability, which avoids having to install loops in the road to trigger the camera.
‘It is difficult to teach software to recognise a car model,’ Williamson remarks. ‘Humans generally recognise makes of cars quite easily, but it’s difficult to classify this for software.’ Manto works in the same way as the human brain and is taught what an object looks like through exposing it to lots of examples. ‘Trying to do that analytically, by measuring different parts of the front of a car, would be virtually impossible to achieve; it would be too difficult to specify the parameters that define a model or make of vehicle,’ he says. Manto is proving to be very reliable for this task, with around 99.9 per cent accuracy in its detection rate.
Currently, as part of the traffic module, Manto has been trained to detect make and model on images of the front of cars. Stemmer also plans to train the system to recognise the rear of cars as well.
Williamson believes that more cameras will be released designed specifically for traffic monitoring and which have camera features focused on addressing the problems associated with imaging in difficult outdoors conditions. Multi-CCD cameras, such as those produced by JAI, or being able to switch very quickly between the settings in a camera to take three or four different exposures in rapid succession, are techniques used to gain high quality images in these sorts of conditions. Dalsa Genie HM1400 and HM1400XDR GigE cameras, available from Stemmer Imaging, are also suited to traffic applications. These cameras feature CMOS sensors with no dead pixels providing high image quality – which, in combination with a compact housing, makes them ideal for traffic applications.
Basler’s Tiarks remarks that high speed and good image quality are two parameters that machine vision cameras typically deliver, as these are two of the most important demands for industrial imaging. For speed enforcement, a certain frame rate (about 30fps) is required to detect fast-travelling vehicles.
With regards to future trends in the traffic sector, Tiarks comments: ‘The trend to incorporate more camera features for traffic enforcement and tolling applications to make the cameras more effective and easier to use will continue – but, at the same time, the big traffic system vendors differentiate themselves through the algorithms and software developed in-house.’ Basler works closely with traffic system vendors to supply cameras and imaging expertise for system development. ‘These vendors require robust, standard imaging equipment, as most of the more complex algorithms, for automatic number plate recognition, for instance, will be developed by the company and will constitute its core knowhow,’ he says.
