Capturing a new vision for transportation management and control
It’s no secret that the highways and byways of the world are becoming increasingly congested and overused. Inefficient traffic flow, increases in motor vehicle accidents, and freight transportation delays can all have a significant economic impact.
These include lost time and productivity, additional fuel consumption, and in worse case scenarios increased personal injury and even fatalities. In addition, the growing security inspection applications required to protect critical transportation systems is another factor to consider.
In an effort to stay ahead of the ever increasing demands of managing and controlling the world’s vast network of roads, the vehicles that use them, and the people that rely on them, transportation systems specialist are turning to computer vision for help. There are numerous software and hardware considerations that need to be considered when deploying a vision system for transportation applications, for the purposes of our discussion here we will focus on camera selection and matching camera requirements to various transportation computer vision applications.
A transportation vision system can provide information ranging from overviews of traffic conditions to details on specific vehicles. In traffic monitoring, for instance, a vision system looking at highway lanes observes the traffic moving within its camera’s field of view and uses simple object detection software to count the number of vehicles passing through per unit of time. The flow information from many such vision systems yields a realtime, wide-area description of traffic conditions.
The central computer of an intelligent transportation system (ITS) can then take this description and use it to manage traffic flow through on-ramp metering, the timing of signal lights at intersections, the changing of lane directions, and the use of driver information signage. The intelligent transportation system could also automatically detect localized traffic stoppages that might indicate an accident, send out alerts, and advise emergency response vehicles of routes to take to avoid congestion.
Adding Eyes for Traffic Law Enforcement
Information from a vision system can support the creation of a virtual toll booth. The vision system captures the image of a vehicle passing through a checkpoint and uses automatic license plate recognition (ALPR) software to identify the vehicle. An intelligent transportation system can then use that identification to send the vehicle’s registered owner a bill for the toll charge.
Vision systems can even provide information to aid in vehicular law enforcement. At traffic signals, for instance, cameras capture images of vehicles that enter an intersection against a red light. These images then serve as triggers for sending out traffic citations to the offending drivers.
Traffic speed laws are equally open to vision system enforcement. One type of system uses a radar or sonar sensor to measure vehicle speed and triggers image capture of vehicles exceeding the set threshold. Another type of system uses automatic license plate recognition to identify vehicles passing a checkpoint, capturing multiple images and using motion detection algorithms to measure the vehicle’s speed. A third type of system uses two different checkpoints and measures a vehicle’s transit time between checkpoints to calculate average speed. All three types of speed enforcement vision systems can simultaneously serve a traffic monitoring application, as well.
A vision system equipped with automatic license plate recognition software can even support law enforcement by scanning the highways for specific vehicles, an operation known as 'fishing'. As the patrol car moves through traffic, an on-board camera with local processing can be continually extracting license plate numbers from vehicles within its field of view. It can then compare those plate numbers with watch lists distributed wirelessly from dispatch to the patrol cars. This extra set of eyes in the patrol car can be configured to look out for vehicles reported stolen or otherwise implicated in a crime, then point out such vehicles of interest to the officers on patrol.
Providing Access Control and Threat Detection
To support site security applications, vision systems can be equipped with both license plate and facial recognition software. They can then automatically determine if a vehicle should be granted access through a security gate by checking the license plate and matching it to the driver. A line scan camera in the roadbed of a checkpoint can enhance security against terrorist actions with minimal impact on traffic flow by examining the vehicle’s undercarriage as it passes the checkpoint. If the vision system detects anomalies that might indicate the presence of explosives, it can alert guards to take appropriate action.
These various traffic monitoring, law enforcement, and access security applications all place slightly different demands on the vision system, and matching the camera to the application’s requirements is essential for obtaining the desired performance at the right cost. Cost is especially critical in applications such as traffic monitoring that need many installed cameras in order to be effective.
Matching Camera to Environment
Vision systems for transportation will typically require that cameras be installed out in the open. Therefore, the first factor to consider is the environment in which the camera must operate, and if it will be exposed to the elements. Cameras may be subject to a wide range of temperatures, fog and precipitation, and wind-induced vibration. Unless a camera is built to withstand such conditions during years of exposure, it will need to be protected in an enclosure.
Another environmental consideration that can affect camera choice is the image area’s illumination. Installations that depend solely on natural daylight and conventional nighttime highway lighting will need a wide dynamic range along with features such as automatic gain control to provide reliable image capture under such wide-ranging conditions. Supplemental illumination such as floodlights can reduce the demands on the camera, but can present a safety hazard unless they are designed to avoid affecting the driver’s vision. Using infra-red illumination that is invisible to drivers can avoid safety concerns, but calls for cameras with special sensors.
A third consideration is the camera’s connectivity to both power and communications. Most transportation applications require external power to the camera because battery-powered installations typically present a significant and expensive maintenance challenge. The exceptions are the vehicle-mounted systems, which can draw from the vehicle’s battery.
Connectivity to image processing and data-storage elements is also required in many applications. Even those cameras with built-in processing and storage usually need a connection for external control and data transfers. Often, such connectivity utilizes networking technology, either wired or wireless. Ethernet is a particularly attractive connectivity option as it is able to provide control, data transfer, and even power for the camera and can readily interface with wireless network transceivers.
Camera Performance and Connectivity Needs Vary with Application
Once the constraints of environment are addressed, most remaining camera selection criteria depend on the application’s specific need for resolution, performance, and functionality, and can vary significantly. The following is a brief review of some of the more common applications and considerations for camera selection.
In a Traffic-Monitoring application, for instance, the camera can be relatively simple. The camera only needs enough resolution and speed for the computing element to perform simple shape recognition and counting as cars pass by. The cameras can typically work in monochrome with available lighting because they need only to distinguish between a vehicle and an empty road. A USB type connection is often sufficient for power and communications connectivity.
Toll enforcement is a little more demanding. Toll gates are typically well-illuminated at night, so the camera does not need extended dynamic range. Further, traffic through toll gates are subject to speed restrictions, so the camera does not need to be ultra-fast. Image processing and communications needs are likewise minimal as the camera only has to capture and store an image of the vehicle as it passes through the toll gate. Image processing functions can be handled off-line working with the stored images.
The toll-enforcement camera does, however, need high enough resolution that automatic license plate recognition is reliable. The camera also needs to include colour operation, and must have a wide enough field of view to capture the entire vehicle’s image. In order to successfully prosecute, law enforcement requires proof of a vehicle and driver match to the license number that the vision system captured. Full-vehicle colour capture is necessary for the image to meet the legal requirements of such proof.
Red light enforcement has similar requirements to toll enforcement but is even more demanding of the camera. The camera must offer wide dynamic range and automatic gain control in order to capture usable images in lighting conditions that range from full daylight to night-time with only a corner lamp post for illumination.
The image capture speed needs to be fast enough to react to the triggering signal indicating a car has violated the red light and to capture an image without motion blur when the vehicle is travelling at high speed. Because real-time image processing is not needed, though, connectivity requirements are minimal.
In vision systems for Speed Enforcement, end-user preferences will dictate whether cameras monitor single lanes or provide a wide view to cover multiple lanes. Both approaches have essentially the same requirements for camera speed and resolution. The major determinant of camera requirements in this application, then, is the way in which the system determines a vehicle’s speed. If a radar system triggers the image capture, for instance, the camera may only need enough resolution and speed to capture and store an image suitable for post processing. Communications requirements are similarly minimal as the vision system does not need real-time connectivity to a network.
If the vision system itself must make the speeding determination, however, camera speed requirements change significantly. The camera must be able to capture multiple images of the vehicle as it passes through the field of view at highway speed. The multiple images are needed so that the system can measure vehicle movement between images and thus calculate speed.
A speed-enforcement vision system may also need enough computational power to perform automatic license plate recognition in real-time. This real-time recognition is necessary whether or not the vision system is doing a spot speed measurement at a single checkpoint or an average speed determination between two checkpoints. In the single checkpoint installation, the plate recognition is needed to ensure that it is the same vehicle in two successive images before the system makes its movement measurements.
In a multi-checkpoint speed enforcement system, real-time automatic license plate recognition along with network communications ability is needed. The vision system must quickly forward both the license plate number and a time-stamped image to a central database. When a second checkpoint camera detects that same plate, the enforcement system can search the database for the earlier image, and then quickly compare time stamps to determine if the average speed between checkpoints violates posted limits. Such comparisons are impractical with offline image processing as the data volume that must be handled for multiple checkpoints is excessive.
Like speed enforcement systems, car-mounted vision systems 'fishing' for vehicles of interest while on patrol need real-time automatic license plate recognition. Such systems would also benefit from high-speed image capture to minimize the impact of the vehicle’s continual motion and bounce. The vision systems do not need real-time connectivity, however, as they can work with a locally-stored database of license numbers. The connectivity needed to update this database can come from the patrol vehicle’s existing communications channels. Such vehicle-mounted vision systems will most likely need custom camera designs, to address installation restrictions.
Security Needs Multiple Camera Types
Two entirely different types of cameras are needed in access control and security inspection applications. The capture of images for driver face and license plate recognition requires an area scan camera similar to those used in red light and toll applications. For undervehicle inspection, however, a line scan camera may be the best choice.
Unlike conventional area scan cameras that capture the image of a two-dimensional area, line scan cameras only capture a thin, one-dimensional slice. As an object moves across the camera’s field of view, however, a properly timed series of such slices will assemble into a traditional twodimensional image of the object. Line scan cameras provide some of the fastest image capture rates at the highest resolutions available and are perfect for obtaining fine detail under difficult conditions.
A line scan camera is ideal in security inspections because it can build an image as the vehicle moves past the checkpoint (thus not impeding traffic) and has no inherent limit on image length. It can capture an entire vehicle in a single image regardless of vehicle size. Working with a single image helps reduce system cost because it simplifies and speeds the image processing needed for the system to make its threat determination.
The cameras in these access control and security inspection systems must offer high-speed connections to an external computer, though, because high-performance image processing is needed in order to perform face and threat recognition in real time. Real-time image recognition is essential for determining if there are anomalies in the vehicle undercarriage and alerting guards before the vehicle has left the checkpoint.
Extensive Vision Technology Available
The wide range in technical and user requirements for these various transportation vision systems means that developers will need a broad selection of imaging technology from which to choose. Teledyne DALSA has a substantial camera and image sensor technology base that offers such a selection, ready to deploy area and line scan camera technology with all the key features needed in transportation applications.
The company also has system design expertise that stands ready to support developers in addressing the diverse needs of transportation vision systems.
