Investigations are underway into using hyperspectral imaging to screen for a host of medical conditions, as Abigail Williams finds out
Hyperspectral imaging (HSI) is emerging as a key underlying technology for a number of next-generation medical systems in applications ranging from protein detection for screening and diagnosing Alzheimer’s disease, to oxygen monitoring and tissue-type identification.
One frontrunner in the development of HSI technology for use in medical applications is the Leuven, Belgium-based institute Imec, which is working closely with medical experts in a variety of fields to develop new instruments, or to upgrade existing tools with added functionalities enabled by HSI. As Wouter Charle, Hyperspectral Imaging Technology Manager at Imec, explains, initially, these solutions can have a supportive function, but will eventually evolve into solutions that “can offer assistance for increased precision and efficiency, or even perform repetitive, routine tasks”.
“With hyperspectral cameras becoming more and more compact, opportunities to integrate them onto various medical tools can be found everywhere,” he says.
One project Imec has been working on is focused on real-time video detection of proteins or oxygen levels in blood. In the past decade, it has become clear that blood will show different signals depending on levels of oxygen saturation, which can be detected through HSI – both in post-processing, and in real-time. According to Charle, the first start-ups offering the necessary algorithms to provide real-time oxygenation, blood volume and true-colour RGB imaging to surgeons are currently emerging in the market.
“Current miniaturised and integrated surgical cameras can only show black-and-white real-time images, which provides less information about the tissues surgeons are looking at, and gives them less means to navigate the body,” he says. “We have been working with medical researchers to integrate our machine vision chips and cameras on existing optical instruments.
“This backwards compatible approach is more cost-efficient, as we’re starting from existing platforms and adding new features onto them; whether it be a new type of sensor or specific filters. We can scale down sensor chips and cameras to fit onto the tip of an endoscope camera, for example – not just to inspect and perform surgery on kidneys, but also on lungs,” he adds.
According to Charle, other areas where HSI can play a part include exoscopy – optical systems outside the body that give surgeons an extra pair of eyes – laparoscopy for abdominal interventions, and oral cavity scanning to detect cancer in the mouth or throat, or for dentistry.
“The next step would be to develop cameras that can distinguish different types of healthy tissue, such as muscles, ligaments, bones and organs – to ultimately detect cancerous tissue. The more accurately cameras can detect unhealthy tissue, the more precisely surgeons will be able to remove growths, with minimal damage to healthy tissue. Realising such precision with hyperspectral cameras will be the holy grail,” he adds.
Screening for Alzheimer’s
Imec has also collaborated with the Flemish research organisation Vito, as well as university KU Leuven and UZ Leuven hospital on retinal imaging for the detection of certain proteins that are indicative of Alzheimer’s disease. Even though Alzheimer’s is a neurological disease, the eyes can give clues to help with diagnosis.
“We’ve developed a snapshot camera that can detect accumulation of two proteins – amyloid-beta (Aβ) and Tau – in the eyes in a very early stage of disease development. In the next phase of our collaboration, we are making a few practical improvements to the camera: creating a new optical instrument to provide a larger field of view and enable imaging at lower-light intensities, and adding a more recent chip with wider bands and higher spectral contrast to detect the proteins more accurately,” says Charle.
As Lies De Groef, Assistant Professor at KU Leuven, explains, the retina is an integral part of the central nervous system (CNS), and thus “closely related to the brain and spinal cord”. Thanks to the transparent ocular media and the availability of an array of non-invasive, high-resolution retinal imaging techniques, the eye provides what she describes as a “unique window into the CNS”.
As such, retinal imaging may be used to understand what disease processes are ongoing in the brain, and retinal biomarkers have become a field of intensive research. One application of retinal biomarkers is their use for non-invasive diagnosis of Alzheimer’s disease. There is accumulating evidence, both from animal studies and in Alzheimer’s patients, for the presence of Alzheimer’s disease hallmarks in the retina – and De Groef and colleagues are exploring the use of different imaging techniques, including hyperspectral retinal imaging, to detect the retinal changes related to neurodegeneration and protein (amyloid, tau) aggregation in vivo.
“Post-mortem studies in both animal and human retinas, and in vivo studies in rodents, have shown that hyperspectral retinal imaging can detect spectral changes in the 460-570nm range that seem to be caused by the presence of retinal amyloid aggregates. A second important biomarker for Alzheimer’s in the eye is thinning of the retinal nerve fibre layer. This can be studied by optical coherence tomography (OCT), and is a sign of neurodegeneration,” says De Groef.
“The current belief is that by combining multiple biomarkers that each cover different hallmarks of the disease – in this case hyperspectral imaging to measure amyloid aggregation, and OCT as a read-out for neurodegeneration – one may come to a multimodal approach for improved diagnosing and monitoring of Alzheimer’s disease,” she adds.
Pilot study
In 2019, De Groef and colleagues ran a pilot study with 17 clinically probable Alzheimer’s patients, seven biomarker-proven Alzheimer’s cases and 22 controls. The study was performed at the Ophthalmology Department of the University Hospital UZ Leuven, in collaboration with the UZ Leuven Memory Clinic. The team found that classification models integrating HSI and OCT data could discriminate between Alzheimer’s subjects and controls with an accuracy of approximately 75% in a nested leave-one-out cross-validation.
“The hyperspectral information was the main driver for this classification result, yet classification accuracy improved by including OCT data,” says De Groef.
In parallel, researchers in the Biology Department at KU Leuven also investigated the use of HSI in preclinical research of Alzheimer’s mouse models – and demonstrated that hyperspectral imaging can also be used to quantify retinal amyloid in animal models and post mortem tissues, underscoring its potential as a biomarker for Alzheimer’s diagnosis and monitoring.
A snapscan VNIR camera shows oxygenation levels of a hand with one tied finger. Credit Imec
“Retinal imaging techniques thus offer unique opportunities for drug discovery and fundamental research into Alzheimer’s disease, and have a high translational value given that the same technologies can be used in mice and humans,” says De Groef.
More generally speaking, De Groef reveals that HSI technology is also currently being used for several other medical applications, including in ophthalmology – where an HSI-based technique called retinal oximetry is used to measure blood oxygenation in the retinal vessels, providing valuable information in the diagnosis and monitoring of ocular diseases like diabetic retinopathy, age-related macular degeneration and glaucoma. Other medical applications include perioperative support (helping to delineate the field of surgery), measurements of cytochrome-c, cholesterol, melanin and haemoglobin, and the detection of neoplasms.
“As hyperspectral imaging is a label-free technique, it holds great potential for in vivo applications. However, it may also be used in pathology research, where it would then circumvent the use of antibodies, tracers and other labelling techniques, and thereby increase speed and cost efficiency,” says De Groef.
Snapshot camera
Another interesting example is a snapshot hyperspectral camera developed by Sweden-based Mantis Photonics for use in a range of ophthalmology applications. As Diego Guénot, Co-Founder and Chief Technology Officer at Mantis Photonics, explains, the eye is in constant motion – particularly the retina at the back of the eye – so ophthalmologists need to capture images of it “extremely quickly”. The technology can be used for the diagnosis of several eye diseases – including glaucoma and diabetic retinopathy – but also to diagnose neurodegenerative diseases such as Alzheimer’s.
“We are about to start two clinical trials, one with patients affected by glaucoma, and another one with patients affected by Alzheimer’s,” says Guénot.
“For glaucoma, our aim is to measure the blood oxygenation by analysing the reflected spectrum of the eye veins and arteries. Blood oxygenation is an indicator of the disease.
“For Alzheimer’s, we aim to distinguish a small spectral shift between healthy patients and patients affected by the disease. This spectral shift is believed to be caused by the presence of amyloid beta, which is an early biomarker of Alzheimer’s,” he adds.
The camera works by shining a light on an object – here, a retina – and the filtering and reflecting of the light using a collimating element. A grating and a spectrometer are then used in combination to disperse and focus the light onto a CMOS sensor.
“In this way, the incoming spectrum is displayed for each microlens. From this, we can either reconstruct the images for each wavelength – or at a single point we can extract the spectrum and, for example, measure the blood oxygenation,” says Guénot.
More generally speaking, Guénot observes that, although the use of hyperspectral imaging technology in medical and healthcare applications is not yet commonplace, it continues to generate a great deal of interest.
“To the best of our knowledge, hyperspectral imaging is not yet an established technique in medicine, but there is an ever-increasing interest in it. More and more groups are investigating it for a large number of conditions,” he says.
“Broadly speaking, hyperspectral imaging is used to measure functional changes, as opposed to the structural changes seen by standard cameras or other imaging techniques. In practice, the main application is for skin imaging, and more specifically detecting skin cancer.”
Moving forward, Guénot believes that AI has a major role to play in hyperspectral imaging, “since it is not a type of data that the human brain is adapted to interpret”.
Diego Guénot’s retina imaged at different wavelengths with a Mantis Photonics camera. Credit Mantis Photonics
“More and more people are combining AI with hyperspectral imaging in many applications – in medicine, but also in agriculture or industrial sorting,” he says.
“In the case of Mantis Photonics, the signal caused by the Alzheimer’s biomarker is very small, while the variation in the retina varies a lot from one patient to another. We will therefore need AI to analyse the data.”
Artificial Intelligence
Moving forward, De Groef reveals that the next steps in her and her colleagues’ research include a larger, multi-centre, longitudinal clinical trial with Alzheimer’s patients in different stages of the disease. Through this study, the Leuven team wants to clarify whether hyperspectral imaging may also be used to diagnose Alzheimer’s in the very early, presymptomatic stage of the disease, and thereby open up an earlier time window for effective treatment.
The Leuven-based research group is also conducting a series of experiments in cellular and animal models with different neurodegenerative diseases, to assess the specificity of the HSI signature. This will clarify what forms of amyloid are being measured by HIS, and whether it may also be used to detect other protein aggregates found in other neurodegenerative diseases, such as tau and alpha-synuclein.
“Given the complexity and the vast amount of data that we gather via HSI, we are looking at AI to improve our diagnostic models. One of our future goals is to further expand our multimodal approach and include more biomarkers of Alzheimer’s in one classification model. AI technologies can help maximise information extraction and force breakthroughs in this field,” she says.
Meanwhile Charle reveals that, in order to enable application research, Imec has CE certified its snapscan VNIR camera to allow integration on surgical microscopes, which will be useful for clinical research surgery. By pairing the data from high-resolution recording of surgeries with traditional analyses by medical specialists, he explains that the Imec team can train AI systems to detect cancer in HSI data.
“And of course, HSI cameras create an enormous amount of data, which will need AI to interpret and translate into useful information,” he says.
The key challenge in all of this is bridging the two worlds of engineering and medical expertise. Translating the medical needs to technical solutions has been one of the more intensive aspects of developing new medical instruments. Cross-disciplinary involvement is a must to develop a sensor ecosystem for robots, or even cobots, that will allow better healthcare,” he adds.
