Hardware accelerators such as graphics processing units (GPUs) and tensor processing units (TPUs) are rapidly emerging in artificial intelligence (AI) as faster processors for applications in computer vision, natural language processing and robotics.
Currently, commercial accelerators are implemented in electronics and suffer the limitation of chip architectures’ processing capacity, whose processing speed lies in the hundreds of MB/s.
This capacity permits the real-time processing of machine vision information limited to only a few conventional colours of red, green, and blue (RGB) channels.
This problem is severe and does not allow the acquisition of important information beyond the primary colour bands, hindering new machine vision applications from emerging. Researchers are therefore exploring alternative technologies, such as optical computation, to overcome this problem. Optical computation directly processes the information in a light beam without converting it into a digital electronic signal.
This approach has several advantages over electronics:
Speed: Light information can be processed as fast as the travelling speed of photons, at the rate of hundreds of TB/s, significantly faster than electronic information could ever be.
Efficiency: Optical computations require significantly less power than electronic ones. This property relies on the fact that optical signals do not generate as much heat as electronics and reach long distances with minimal energy loss.
Parallelism: Because light can travel through multiple pathways simultaneously, optical computations can perform millions of operations in parallel at practically no cost.
Very recently, our team of researchers of the Primalight group at Saudi Arabia’s King Abdullah University of Science and Technology (KAUST) developed the first form of ultra-thin optical accelerator hardware for deep learning. This system, called Hyplex, uses light-matter interactions occurring in universal light encoders, which represent suitably engineered nanostructures with characteristic features as small as 20nm.
Hyplex technology can be integrated directly within any conventional camera and serve as a lightspeed feature extractor for spectral encoding, depth-estimation or polarisation measurement. Despite the early development of optical accelerators, Hyplex already recently demonstrated quantitative advantages in hyperspectral imaging (HSI), a field of machine vision currently attracting tremendous interest. In contrast to traditional imaging methods that only capture information in a narrow range of RGB colours, HSI acquires data across a continuous range of wavelengths. This information provides key signatures about any object’s chemical and physical properties in a scene, opening key applications. In precision agriculture, HSI can detect plant stress and automatically segment and classify vegetation maps; in medicine, HSI detects and classifies various diseases, including cancer; in environmental science, HSI can monitor and detect pollution, to mention a few examples.
Hyplex technology can be integrated directly within any conventional camera
Currently, the best HSI available requires significant investment (in the range of $20,000- $200,000) and is affected by a slow speed of data processing (minutes). It necessitates the use of significant amounts of computational resources to post-process the large amount of data generated, while being incapable of generating high-resolution videos. The optical accelerators at the core of Hyplex enable it to address these issues. Unlike present hyperspectral systems, Hyplex does not use spectrometers or filters and can acquire and process hyperspectral videos in real time at the same rate and with the exact resolution as a modern digital camera, for example 12MP at 30fps or better. Hyplex has been validated against modern spectral reconstruction and hyperspectral segmentation benchmarks. In all of these tests, it outperformed all state-of-theart alternatives because none of these systems employ optical accelerators.
Pixeltra, a KAUST start-up with representative offices in both the UAE and Saudi Arabia, commercialises Hyplex technology. Recent developments of Hyplex are in precision agriculture using unmanned aerial vehicles (UAVs, or drones) to capture high-resolution hyperspectral images of crops. These images are analysed using machine learning software to identify diseases, pests, and nutrient deficiencies. The information can then be used to make more informed decisions about planting, fertilisation, and other management practices, ultimately increasing crop yields and reducing costs.
Another development is DermaPlex , a new portable and non-invasive HSI device for the realtime diagnosing and classification of skin cancer. DermaPlex facilitates skin cancer diagnosis by capturing a detailed chemical profile of the tissue, analysing the texture and colour of skin lesions, and monitoring skin cancer progression over time. This system provides non-invasive imaging that eliminates the need for biopsy and when combined with AI, can drastically enhance diagnostic accuracy.
Hyplex is being deployed in precision agriculture using unmanned aerial vehicles to capture high-resolution hyperspectral images of crops
Driven by these technological achievements, we expect considerable growth in interest in HSI and video understanding for both industrial and research areas. Hyperspectral systems based on optical accelerators overcome the limitations of multispectral imaging, allowing them to tackle problems in which the information is sparse across a wide spectral range. These include remote sensing, medical diagnostic, precision agriculture, and security. The applications portfolio of optical accelerators such as Hyplex is not limited to HSI. One of the main advantages of optical accelerators is that they can perform matrix multiplications – a fundamental operation in deep learning – a few orders of magnitudes faster than electronic devices. This feature can significantly speed up the training time of neural networks and enable the training of larger and more complex models not achievable by current electronics.
Optical accelerators can be used as both standalone devices and in combination with other hardware accelerators, such as GPUs, TPUs and field-programmable gate arrays (FPGAs), to increase the performance of deep learning algorithms further. With sufficient development, optical accelerators can enable new future pathways for machine vision applications using deep learning, increasing the speed and efficiency of machine learning algorithms in more complex models that are not possible with conventional electronics today
Professor Andrea Fratalocchi is head of the Primalight research group at KAUST, and cofounder of Pixeltra
