Being able to see what's going on in the body is incredibly useful for doctors and other medical professionals. X-ray, ultrasound, magnetic resonance imaging (MRI) and computed tomography (CT) all provide diagnostic evidence and can help doctors reach a decision on the best course of treatment. Surgeons, for instance, can study images of the damaged or diseased area in order to gain a better understanding of the nature of the problem, what surgery is likely to achieve, and how best to approach the operation. Machine vision can also play an important part during an operation, with cameras guiding a surgeon’s incisions.
One way to obtain images of internal organs is to use an endoscope: a miniature camera positioned on the end of a flexible tube that can be fed into the body. Endoscopic exploration is limited to the size of the endoscope, which varies. The currently available ‘chip on tip’ endoscopes are generally no smaller than 2.4mm in diameter.
When carrying out an inspection of the lung, for instance, this means that endoscopes can travel only a certain distance down the bronchi. For inspection of tissues lying deeper in the lung, where passageways are less than 1mm, clinicians use fibre bundles – a bundle of around 3,000 optical fibres each transporting one pixel of information. Often though, the images generated by fibre bundles do not have high enough resolution to make clinical decisions and the cost is high as fibres break easily.
Awaiba, based in Madeira, Portugal, develops image sensors for industrial inspection, medical endoscopies, high-speed video systems and automotive on-board cameras. It has developed a small camera for disposable endoscopes. Measuring less than 1mm in diameter and using CMOS technology, the NanEye endoscopic camera employs wafer level optics and packaging to reduce the size and cost of the product. ‘The low cost of CMOS technology ensures the NanEye is cheap to manufacture and can therefore be used as a disposable item, sidestepping the problems associated with sterilising endoscopic equipment,’ comments Martin Wäny, CEO of Awaiba.
‘One application of the NanEye camera is as a visual aid for positioning locator devices used in radiotherapy,’ explains Wäny. To minimise the destruction of healthy tissue in patients undergoing radiotherapy, doctors place a locator device close to the tumour as a target for radiation. ‘Positioning the device can sometimes be simply trial and error for surgeons, and introducing a camera into the procedure provides obvious navigational benefits,’ he says.
‘There is a big demand for minimal- or noninvasive surgery in the medical industry and miniaturisation of equipment using vision will continue to find uses in novel applications,’ notes Wäny. ‘Providing navigation for surgeons is one such area where the miniaturisation of machine vision is being exploited.’
Dr Dimitar Deliyski pictured with a high-speed videoendoscope, a device used to explore the movements of the human vocal folds. Image courtesy of Dr Deliyski and Vision Research.
A further endoscopic application is under development at the University of South Carolina’s Arnold School of Public Health. The five-year project, led by Dr Dimitar Deliyski and currently in its second year, aims to develop a new methodology for functional evaluation of voice disorders. The study looks to explore the movements of the human vocal folds using a procedure known as laryngeal high-speed videoendoscopy (HSV).
Attempts were first made to use high-speed motion pictures for visualising the vocal folds in the 1930s, following the introduction of the first high-speed camera by Kodak. ‘The early devices were highly impractical as vocal folds vibrate extremely quickly – in excess of 1,000 times per second during falsettos,’ explains Deliyski. ‘It is only now that high-speed cameras are not only fast enough, but sufficiently sensitive to capture real-time video footage of the vocal folds.’
Vision Research, a New Jersey-based manufacturer of high-speed digital imaging solutions, has supplied its Phantom V7.3 digital camera for use in Deliyski’s research. The camera operates at a rate of 6,668 pictures per second (pps) at full resolution (800 x 600 pixels) allowing real-time footage of the vocal folds to be taken.
Currently, the predominant method for studying voice disorders is laryngeal videostroboscopy. Here, a computer-controlled strobe light illuminates the vibrating vocal folds and a camera captures an image at approximately 25-30pps with each flash of the strobe. This gives an apparent slow-motion video of the event and has been used successfully in diagnosing many voice disorders. However, the video is not a true reflection of the movement of vocal folds as data is missing between flashes of the strobe. ‘If vibrations are not in a regular pattern, which occurs in approximately 50 per cent of patients, then videostroboscopy cannot provide reliable functional information to aid diagnosis,’ says Deliyski.
‘HSV has the advantage over videostroboscopy of being able to capture footage in colour as well as in monochrome,’ says Rick Robinson, director of marketing at Vision Research. However, Deliyski is currently investigating whether there is any diagnostic advantage of using colour over black and white images. ‘The technique requires constant lighting, as opposed to a strobe effect, but to avoid burning the vocal folds the strength of illumination must be kept to a minimum. Using monochrome provides a higher sensitivity and allows images to be taken in lower lighting conditions. Therefore, if there is no significant advantage to recording colour images, then monochrome is preferable,’ explains Robinson.
‘One of the challenges of using HSV is processing all the data efficiently in order to pinpoint important features,’ says Deliyski. ‘Ten seconds of footage, when slowed down to show the vibrations, can last hours, and being able to extract useful information from this will be a vital part of implementing the technique.
‘The project aims to improve the clinical protocol for voice evaluation,’ Deliyski continues. ‘However, there is so much new data being generated that it will take time to process all of it and work out exactly what we’re seeing. Over the next few years we hope to improve our understanding of how the voice works and to use this technique as a tool to aid voice experts in making clinical decisions.’
Delicate surgical procedures require immense levels of concentration and a steady pair of hands. VR Magic, a German camera manufacturer, is developing an optical tracking system to allow orthopaedic surgeons to carry out bone drilling and milling operations with a greater degree of precision. The work is part of the Intelligent Tool Drive project currently under development at the University of Heidelberg.
A segmented MRI scan of the knee showing different regions. Image courtesy of Dr Nick Granville.
The collaborative project is led by Dr Markus Schwarz from the Laboratory for Biomechanics and Experimental Orthopaedics at the University Clinic Mannheim and involves the Orthopaedics and Accident Surgery Centre and the Central Institute for Technical Informatics (both at the University of Heidelberg), VR Magic, and MRC Systems based in Heidelberg.
‘The optical tracking system, which has been four years in development, is currently at the prototype stage and is showing positive results,’ says Thomas Ruf, director of research and development at VR Magic. ‘The overall project aims to aid orthopaedic surgery, but VR Magic’s optical tracking system could potentially be used to carry out other high precision work, such as that found in the electronics industry.’
Cameras track the movements of both patient and doctor, and feed images into a robotic system, which compensates for those movements as well as any vibrations caused by the machinery. To achieve this in real-time, the cameras need to be fast and have a low latency period. The cameras are equipped with a high-sensitivity 1-inch sensor with 1.3 Megapixels and operate at a speed of 100Hz.
An FPGA module (Field Programmable Gate Array) and a DSP (Digital Signal Processor) preprocess the image data in the camera in order to minimise system latency. 3D reconstruction then takes place in the connected host system, and this information is used by the robot to move the instrument onto the path planned prior to the start of surgery.
‘The system can be scaled up if necessary,’ explains Ruf. ‘A minimum of two cameras provide a stereo image of the area being operated on, but further cameras can be integrated into the system to ensure the view is not blocked during the operation.’
Prior to operating, surgeons use medical images to assess the condition and devise a suitable course of action for surgery. However, the images are useless if the surgeon is unable to interpret them correctly. Through his work at Smith and Nephew, a global company developing medical devices for healthcare professionals, Dr Nick Granville was involved in studying MRI scans of the knee, with an aim to provide clearer interpretation of those images for surgeons. He is now an independent consultant.
MR produces greyscale images that can be easily analysed for organs such as the brain, where defined grey and white matter regions are automatically partitioned using image processing software. However, some skill is required to interpret the greyscale images of the knee, or any orthopaedic MR images, as the different components (bone or cartilage, for example) are difficult to segment automatically.
‘X-rays and CT are currently the predominant imaging techniques for analysing injuries relating to the knee,’ says Granville. ‘They are lower cost and produce images in a shorter timeframe, but do use ionising radiation, which MR does not.’
Whilst the level of ionising radiation associated with CT scans is falling due to advances in the technology, MR still provides a safer method of imaging and Granville feels the use of MRI will continue to increase. However, due to the nature of MRI, the equipment is very bulky, and it will be this reason, if any, that will delay the growth of MR as an imaging technique.
Granville uses Matlab software from the Massachusetts-based company, The MathWorks, to analyse MR images of the knee. A reference image of the knee is set up with markings denoting geometric information that show the various parts of the knee. This model is then applied to other images undergoing analyses to obtain a segmented image.
‘Once the image has been partitioned into constituent components, so much more can be done with it,’ notes Granville. ‘Colour coding, measurements of certain areas, labelling – all can help a surgeon make sense of what was once a difficult image to interpret, and aid in a decision on the best course of action for treating the patient.
‘In an ideal world, a scan of the patient’s knee undergoing surgery would result in an automatic recommendation of implant size for the surgeon to consider,’ he says.
This is the goal for developing image analysis software and Granville feels that this could be the reality in the next few years.
Digital Radiography (DR) is an x-ray imaging technique that uses digital detectors instead of a traditional x-ray film. Predominantly, DR detectors are either amorphous selenium-based detectors, which convert x-rays directly into electrons, or amorphous silicon-based detectors, which use light as an intermediary in the conversion of x-rays to electrons.
‘The potential attributes of low cost and high image-quality associated with CCD and CMOS detectors could be hugely beneficial for DR,' explains Behnam Rashidian, product manager, life sciences at Dalsa. 'However, currently CCD and CMOS detectors are only used sparingly in DR as the field of view is limited by the silicon wafer size.'
Dalsa has developed a sensor tiling and butting technology which allows for larger CCD and CMOS detectors by tiling its CCD and CMOS sensor dyes together. The detector is designed for use in DR applications requiring a large field of view, such as full field mammography, which sees a market growth rate of greater than 20 per cent per year. Similar detectors could be used in stereotactic needle biopsy applications, where the images guide the surgeon’s needle during a biopsy.
A dynamic analysis of coronary arteries, taken using Dalsa’s X-Frame CCD series for radiology and fluoroscopy applications. Image courtesy of Dalsa.
‘A further application that could benefit from the sensitivity and speed of CCD detectors is micro-CT technology,' says Rashidian. Micro-CT scans are used in pre-clinical testing of drug candidates in small mammals, which requires high-speed imagery due to the animal’s rapid heart rate and metabolism.
‘CCD technology has the potential to replace traditional flat-panel x-ray detectors, but this transition will occur slowly,' notes Rashidian. 'The medical market is very concentrated and regulated, with only a few major providers of x-ray imaging equipment. This, coupled with a lengthy product lifespan, means any changes in x-ray equipment happen gradually.’
