The surveillance of large, busy areas, such as railway terminals or airports, requires constant coverage and large numbers of cameras. Vision systems that are able to pick out certain behaviour, or to automatically track individuals or vehicles within a complex scene, are hugely beneficial to security personnel, as suspect behaviour can be flagged for analysis where it might otherwise have been missed.
‘The technology surrounding IP cameras developed for the security market is being driven towards running more complex video analytics onboard the camera,’ comments Julie Busby, technical sales director at UK-based camera and vision component distributor Multipix Imaging. Typically complex processing is run on the PC, however, the disadvantage with PC-based analytics is that the video needs to be streamed continuously, which increases the required bandwidth. Carrying out analytics onboard the camera helps to reduce the bandwidth requirements on the system, allowing a greater number of cameras to be connected to a network. Camera manufacturers are focusing on engineering systems that will accomplish more complex analytics onboard without having to move to the PC.
One smart camera solution containing onboard analytics is being developed by Sony, with ScanaFace software from Dutch company Abstract Computing International, as a security measure to provide access control to buildings. The technology detects and tracks faces based on features recognition and is aimed at providing a failsafe against intruders using stolen pass keys to gain access to buildings.
A typical access control solution involves an electronic pass at an entrance for which the employee is required to provide ID, with a camera monitoring the entranceway. An unknown person using a stolen pass will usually try to conceal their face from the camera. One potential application of the smart camera is to analyse the video feed and overrule any valid pass presented if the face is hidden. ‘It is a soft security measure, in that it’s not carrying out full face recognition and matching that to a database of profiles, but simply detecting whether the face is there or not,’ says Allard Blom, chief operating officer at Abstract Computing International.
The camera can also be positioned within a building to capture images of the intruder. ‘The technology is designed to maximise the amount of information available on the person,’ says Arnaud Destruels, product marketing manager at Sony Europe. Often lighting is not sufficient, making it difficult to capture a usable image of the intruder’s face. At the same time, if the person is moving continuously it compounds the problem of positive identification. ‘The smart camera grabs frames from the video feed showing the face from the best possible angle,’ explains Destruels. There is then some post-processing carried out on the images to alter the lighting levels and colour to provide an image of the person that is recognisable and that can be used by the police.
‘The advantage with this technology is that it’s targeted at tracking faces,’ states Blom. ‘The camera knows what it’s looking for, which is different to other systems tracking objects in railway or airport terminals, for instance.’ According to Blom the problem with tracking objects, such as luggage or crowds of people, is that they are, in some capacity, unknown and this can lead to false alarms. The technology can also be used in a number of other applications, including traffic monitoring and retail marketing – identifying the length of time customers spend in certain areas of the shop.
A smart camera solution is currently being developed by Sony, using ScanaFace software from Abstract Computing International, that detects and tracks faces based on feature recognition.
A common video analytic often deployed onboard security cameras is a virtual trip wire. Here a mask, or a virtual line on the screen, is set up and any movement across that line will be picked up, as will the direction of the movement. This can be used to monitor vehicles moving the wrong way down a one-way street, for instance. Escape routes or stairways in buildings can also be monitored in this manner, where, apart from during fire alarms, there should be no one passing through. ‘Such analytics can be regarded as virtual alarms which can be recorded and will also flag the images for further analysis by security officials at a central control room,’ notes Busby of Multipix Imaging. Removing the need to continuously stream the video from a scene where changes or movement are not expected under normal circumstances can also reduce the storage space required.
The ability to process increasingly complex video analytics within the camera is being helped by advances in processing technology. Eutecus, a company with headquarters in California, US producing firmware and software capabilities for high-speed video analytics, has developed a processing architecture embedded in a Field Programmable Gate Array (FPGA) designed for use in intelligent cameras. The Multicore Video Analytics Engine (MVE) is designed for high-speed video analytics and includes algorithms to identify specific events, such as looking for left luggage at airports. Multiple events can be specified, with associated masks that can overlap.
Digital Signal Processors (DSPs), an alternative form of embedded image processing architecture, are limited by the amount of data that can be processed effectively. ‘The effectiveness of processing with DSPs is reduced at high resolutions or when the camera is used to identify multiple events,’ explains Dr Gusztav Bartfai, director of European operation at Eutecus. Security applications, however, are often processing-heavy and require a level of parallelism found in FPGAs, in combination with software, to cope with the complexities and processing demands of security applications. Bartfai points out that these forms of processors are not mutually exclusive and the MVE can be used to offload the DSP so it can deal with other tasks such as encoding more effectively.
‘The need for intelligent processing is expected to increase, as will processing “at the edge”, or within the camera, in security applications,’ comments Bartfai. Intelligent processing relieves some of the responsibility from the operator, as the camera will flag up events for security officials to view. Intelligent recording is also an important aspect, Bartfai notes, and if the data is tagged when stored then it becomes possible to search it. ‘If events can be detected intelligently then this same technology can be used to search intelligently,’ he says. Imaging data used for forensic purposes, for instance, can be searched for a specific time tag to find footage of an incident key to police enquiries.
Bartfai states: ‘The range of applications is growing and this places greater demand on image processing capabilities.’ Cost of installation is also an issue with most security applications and the MVE provides a high-performance, low-cost imaging solution with low power consumption.
Whether video analytics is executed in the camera or at the PC is dependent on the application and the capability of the camera. The amount of processing that can be carried out within the camera is limited and there is a trade-off between the advantages of running onboard analytics and the camera’s performance. While the processing power available onboard intelligent cameras is growing, Busby of Multipix Imaging feels that certain algorithms or processing certain complex scenes will require PC-based analysis for the foreseeable future.
