Artificial intelligence is being used in a new multi-modal image fusion technology that automatically combines the best parts of images from different sources. The technology, jointly developed by NEC Corporation and the Tokyo Institute of Technology, can combine visible images taken by standard cameras with non-visible images taken by specialised devices, such as thermal or terahertz cameras.
Specialised non-visible imaging devices, such as infrared and x-ray cameras, can now be used in an extended range of environments, including night time monitoring under severe weather conditions. However, these cameras tend to provide images with poor resolution and quality in comparison to images taken of visible subjects. To overcome this, non-visible images are often compared or combined with clearer visible images of the same subject; however, in the past this has lead to abnormalities or hazards within the images being overlooked.
'This technology eliminates the need for such manual work, using artificial intelligence to effectively and automatically combine images taken by different cameras,’ said Professor Masatoshi Okutomi of the Tokyo Institute of Technology. ‘This also increases visibility by actively utilising the strong points of each visible image and non-visible image, even when the images are difficult to visualise.’
The artificial intelligence carries out detailed examinations of each image and assess the degree of visibility of each part. The best areas of each image are then automatically extracted and combined in a multi-modal image, taking into account environmental characteristics such as brightness, light direction and obstacles.
‘As this technology enables instant visual clarification, even under harsh conditions, it allows users to make well informed evaluations,’ commented Akio Yamada, general manager of Data Science Research Laboratories at NEC. ‘For example, it can be applied to monitoring systems to assist with night time observations, or for infrastructure inspection devices to improve the detection of interior and exterior abnormalities, such as cracking.’
The artificial intelligence also analyses and detects abnormalities and hazards in non-visible camera images while simultaneously avoiding image breakdowns, such as clipped highlights and crushed blacks. Image analysis is a common use of artificial intelligence and deep learning. In April, system-on-chip manufacturer Socionext and Japanese AI software company Soinn presented results of a trial using deep learning algorithms to assist technicians and detect human errors in medical image handling.
NEC and Tokyo Tech presented the technology on 7 June at the 23rd Symposium on Sensing via Image Information in Yokohama City, Japan.
