European scientists are developing an augmented reality (AR) surgical visor capable of superimposing a patient’s x-ray images and other medical information in perfect 3D unison with their anatomy.
The visor prevents surgeons from having to look away during delicate procedures to view patient data, which could lead to reductions in surgery times and improvements in surgical accuracy, according to the researchers.
The VOSTARS will combine virutal and augmented reality elements to display important patient information to surgeons during delicate operations. (Credit: Medical Realities/PA)
The Video Optical See-Through Augmented Reality surgical System (VOSTARS) will display a patient’s anaesthetic data, heart rate, body temperature, blood pressure, and breathing rates conveniently into the surgeon’s field of vision. The project, which is funded under Horizon 2020, forecasts significant improvement in surgical accuracy and reductions in the time spent under anaesthetic by at least 11 per cent.
‘With this state-of-the-art, highly ergonomic visor, we intend to provide all the information required to improve surgery,’ explained project coordinator Dr Vincenzo Ferrari, biomedical engineering researcher at the department of information engineering at the University of Pisa. ‘The primary goal is to reduce not just surgery times, but also the time spent under anaesthetic and the cost involved in any operation. For the patient, this means saving 20 minutes of every three hours of surgery.’
The visor incorporates small, highly luminous micro displays and LED optical waveguides to project 2D x-ray images into the vision of the wearer. The system will work by capturing a surgeon’s perspective using a head-mounted camera and then merging this footage with real-time patient data. A central processor using advanced registration techniques for surgical navigation will then present a real-time hybrid image on the visor. ‘A clinician can move freely while still seeing the patient, the hybrid x-ray image and all of critical data all at once in a surgical dashboard inside the screen,’ said Ferrari.
The visor will combine two existing AR technologies: video see-through (VST) and optical see-through (OST). OST systems such as Microsoft Hololens grant the user a direct view of the natural environment with computer-generated images superimposed on the user’s field of vision using a semi-transparent mirror. VST systems such as the Occulus Rift submerge the user in the virtual world, seeing through a closed head-mounted display (HMD) together with stereo cameras and screens.
Fabrizio Cutolo, engineer and expert in wearable augmented reality systems, believes neither VST nor OST alone are suitable for operating on a live patient. ‘When operating on a real person, it was clear to us that the benefits of OST and VST systems could be combined to make a hybrid device,’ he said.
‘For something as critical as an operation, we had to have the naturalistic “feel” of OST, while having the fluid interaction of the VST,’ Cutolo continued. ‘Therefore a brand new device had to be made from scratch, rather than extend an existing technology. The VST-OST augmented reality hybrid was born.'
Already three months into the three-year project, VOSTARS aims to have a working prototype of the device ready for May 2018. Initially being trialled on a number of procedures to the head, including maxillofacial (jaws and face) surgery, neurosurgery, ENT (ear, nose and throat) and orthopaedic surgery, the project hopes to be available to end users in three years, with mass production underway by 2022.
Related stories
Heightened reality: VR experts give optics wish list at Photonics West
