In the industrial and scientific imaging sector, there always used to be a battle between CCD sensors, where image quality ruled, and CMOS, for higher frame rates. This is a fight that CMOS seems to be winning, although there are areas in astronomy and life sciences where CCDs are still the only option. All the major suppliers of industrial image sensors are offering advances in their sensor technology, but Sony has released a sensor that combines aspects of both CCD and CMOS in a pixel design, with some impressive performance specifications.
The Sony Pregius sensors are Sony’s first global shutter CMOS sensors – global shutter is considered important for industrial vision as it avoids distortion. Daniel Diezemann, senior vision consultant at IDS Imaging Development Systems, described the architecture for the IMX174 sensor, part of the Pregius family, as ‘like a CCD pixel in a CMOS sensor’.
‘For 30 years, Sony has been the world’s major supplier and manufacturer of CCD sensors. Sony now brings this knowledge to its new pixel design for the CMOS IMX174 sensor,’ he remarked.
The IMX174 has an analogue pixel design similar to a CCD, with the back end basically a CMOS, explained Diezemann. The sensor has the advantages of the CCD pixel – good colour reproduction, good performance, low noise, and high dynamic range with full well capacity – along with all the digital advantages of a CMOS device, so analogue-to-digital converter (ADC), image correction, and the digital output and high speed.
‘You will never achieve this frame rate with a CCD sensor,’ commented Diezemann. The 1,920 x 1,200-pixel resolution IMX174 offers a frame rate of 162fps. He added: ‘I’ve never seen such a good image from a consumer market sensor than with the Sony IMX174. We measured it with the EMVA1288 camera measurement standard in our optic labs and it is far superior in terms of measurement data compared to most other comparable sensors. There is a really huge gap between them. Sony has really mixed up the market with this new sensor.’
High praise indeed, but it is borne out by some impressive EMVA1288 specifications for the sensor – quantum efficiency at 525nm is 76 per cent for the 2.3 megapixel IMX174, dynamic range is 73dB, and the sensor has a low temporal dark noise of seven electrons meaning more signal gain without compromising image quality. ‘By implementing analogue memory at the pixel, Sony has reduced temporal dark noise to levels comparable to a CCD,’ commented Paul Kozik, product manager at Point Grey, which uses the new Sony IMX174 sensor in some of its Grasshopper and Blackfly camera models.
There are, however, other important sensor criteria for industrial imaging, noted Dr Lou Hermans, COO of sensor company Cmosis. He said that the global shutter and low dark noise of the Sony sensor was an advantage for machine vision, but that other performance parameters such as frame rate, signal-to-noise ratio and near-infrared sensitivity were also key specifications.
In addition, there are advantages to having a family of sensors offering standard sizes and electronics to make it easy for camera manufacturers to integrate different sensors into their cameras. Thad Smith, director of business for image sensor products at On Semiconductor, commented: ‘IMX174 can indeed be considered a disruptive technology. The technology demonstrates the possibilities of the new charge-based pixel approach and sets a new benchmark for image quality.
‘However, while IMX174 is a single part in the portfolio, On Semiconductor’s image sensors offer a larger range of solutions for the industrial market. This is a key differentiator. Each of our customers is looking to streamline their own development and find synergies between sensors. Our platform approach, added with a single PCB solution combining multiple packages, is a clear step towards reducing development costs. In addition, Python [On Semiconductor’s sensor range] introduces a speed increase while reducing the region of interest in both directions. This is particularly important for motion analysis applications. The frame rate scales almost linearly with the resolution, and is not limited by line timing.’
According to Diezemann, a camera manufacturer like IDS requires one year from receiving a new sensor to the camera reaching serial production. ‘Sensor manufacturers want their sensors to be integrated in camera designs quickly and in high volumes,’ he said. ‘Sensor manufacturers are now using the same housing, the same pining, the same internal register layout for their new sensors, so it’s very easy for a camera manufacturer to change sensors. This means that time-to-market for new cameras is much faster than years ago.’
On Semiconductor’s Python family of image sensors, which was launched at the Vision trade fair in Stuttgart, Germany, last November, covers resolutions ranging from VGA up to 25 megapixels, and offers low noise global shutter pixels and high-speed. The entire family of sensors can be supported from a single PCB, resulting in a hardware-scalable product family where all sensors have the same data, control and optical interface.
In terms of speed, machine vision CMOS sensors are available offering up to 500fps, which means systems are able to inspect fast-moving machinery like car engines or webs of material running at high speed. A fast sensor can also give high dynamic range by combining images taken at different exposures.
Machine vision sensors also often now have features like sequencer and multi-area of interest (AOI), remarked Diezemann. Multi-AOI increases the frame rate with windowing – so that, for tracking a package, for example, the engineer would set up four areas of interest to track the four corners of the package. Sequencer allows the sensor to change registers automatically from one image capture to another – the first image might be a 10ms exposure, the second image 5ms and the third 1ms. So, with one trigger, three images are captured at different exposure times to combine them into a high dynamic range image.
Life sciences
The life sciences market always used to be dominated by CCDs because of their superior image quality. Now, according to Dr Ruediger Bader, a sales engineer at Hamamatsu Photonics UK, CCD sensors are being replaced by scientific CMOS sensors for life science imaging.
The advantages of sCMOS sensors are: high speed in the hundreds of frames per second region without having the higher read noise typical for CCDs; larger sensor areas for a wider field of view and higher resolution; and a high dynamic range. Scientific CMOS sensors are also cost-effective compared to other low light detectors such as EMCCDs.
Dr Bader explained that new sCMOS sensors are large enough to be divided into two separate image halves, with the light from two distinct types of cellular structure captured in each half via special image-splitting optics, such as Hamamatsu’s W-View Gemini. The images can then be super-imposed over each other to provide simultaneous dual-wavelength imaging. The user can control the acquisition time of each sensor half independently of the other, and thereby correct for any intensity differences coming from the two fluorescent dyes, noted Dr Bader.
While sCMOS sensors are ideal for life science imaging, Diezemann commented that the new Sony IMX174 sensor is close to sCMOS sensors in terms of image quality, meaning it too can be used for these low light microscopy applications.
Looking to the future
The trend in microscopy imaging is for large sensors, 2/3-inch or one-inch sensors, to get a wide field of view and higher resolution. For machine vision, however, smaller sensors are preferable to keep the costs down.
Diezemann said that the next two years will be great for IDS and for other camera manufacturers. ‘Sony has a new sensor technology with smaller pixels in their roadmap; they’re going higher resolution, full HD, 4k cinema resolution and higher frame rates, 200fps and more. With these sensors you can go into new applications and new markets, which couldn’t be solved years ago,’ he remarked.
‘It’s not possible at the moment to produce a very small global shutter pixel. I expect that in one year there will be in mass production a global shutter pixel in the 3µm range. This means we would have full HD and 4k cinema resolution in cameras with a C-mount lens. This is very important for our market. Machine vision needs a small sensor, 2/3-inch or 1/2-inch is best, and we need small pixels, because everyone wants high resolution.’
Cmosis is moving towards advanced CMOS technologies, according to Dr Hermans, which were originally developed for very small pixels used in mobile phone applications, and applying these to machine vision and other industrial and professional imaging applications.
Kozik at Point Grey stated that industry consolidation within the image sensor market ‘should help smaller companies increase innovation when compared to Sony, who has been the leader in this regard for many decades’. This has been the case in the last year, with On Semiconductor buying Truesense Imaging and Aptina, e2v purchasing AnaFocus, and Cmosis acquiring Awaiba.
In terms of Sony’s new Pregius global shutter CMOS sensors, most expect it will be a popular sensor for machine vision and scientific imaging. Diezemann said: ‘The CCD market is declining. I predict that there are less than 100 million CCD sensors produced worldwide. Sony knows that CCDs are declining and so the best way is to have the advantages of CCDs like colour reproduction and pixel performance, and to move that into a CMOS sensor. Sony has its own fabs and is the only company that can produce the silicon layer needed for this pixel performance.’
It will be interesting to see the uptake of Sony’s technology and developments by other sensor manufacturers in the market.
At Photonics West in San Francisco, which took place from 7 to 12 February, Adimec and LLTech were presenting a full frame optical coherence tomography (FFOCT) camera. The device was developed as part of the European CAReIOCA project to provide pathologists and surgeons with an endoscopic camera to speed up the assessment of cancerous tissue.
The CMOS sensor at the heart of the camera was developed by Cmosis and according to Guy Meynants, CTO of the company, the sensor collects 20 times more photocharges than the sensors traditionally used for OCT applications.
Kevin Van Esbroeck, Cmosis project manager for the sensor development for CAReIOCA, commented: ‘One of the key things for this application is having high speed in combination with a high full-well capacity and global shutter.’
The camera has been integrated into a handheld endoscope, which takes two images and analyses the difference between them. There is only a very slight difference in the optical signal between the two images, explained Meynants, and so to get a good signal-to-noise ratio and detect the difference between these two images it requires a high full-well capacity.
The other aspect is the speed of the readout of the pixel array, which has to be fast enough to freeze any movement from either the patient or the surgeon.
Sensors for OCT traditionally have a full-well capacity of around 100,000 electrons, whereas the Cmosis sensor’s full-well capacity is two million, so 20 times higher. It has a large pixel, 12µm, in order to store enough charge inside the photodiode. The two-megapixel global shutter area scan sensor captures sequentially two full field interferometry images with different optical path lengths of the tissue. The calculated OCT image represents a slice of the tissue for analysis.
Pathologists would normally have to study cancerous tissue from a biopsy under the microscope. It is hoped, however, that this device will speed up the assessment of cancer cells by giving surgeons and pathologists faster and more detailed access to diagnostic data.
Adimec and LLTech have integrated the camera into the FFOCT device. Now, according to Meynants, two hospitals are planning clinical trials using the instrument. The project partners are also analysing the OCT images to develop protocols to recognise certain cancer cells from the images.
