Understanding Line Scan Camera Applications
A line scan camera has a single line of pixels that image across an object. To build up a two-dimensional image of the object, either the camera or object is moved perpendicular to the line of pixels. This might seem like a complicated way to image an object compared with “frame cameras” that take two-dimensional images (a “frame”). However, when the object is large, continuously moving, or the task needs perfect or high resolution imaging, a line scan camera is often a much better choice than a frame camera.
Suppose a machine vision system is to inspect long rolls of material, such as paper or plastic, for defects. A resolution of 16,000 pixels across the material is needed to detect these defects. The material is unrolled under the camera and the material’s speed is measured by a rotary encoder connected to the unrolling mechanism.
You could use four large-format frame cameras mounted side-by-side across the web of material, but this has a number of disadvantages. First, large-format frame cameras usually have defective pixels that are hidden by filling in a pixel value from neighboring pixels. This can hide the defects you are trying to detect. Second, the frame camera’s exposure time must be short to prevent blurring due to the material’s motion. A short exposure requires high intensity lighting over the entire imaged area to get an acceptable image. The rotary encoder output triggers synchronous image acquisition for all cameras. Last, removing offset and gain variations in pixel responses – necessary to detect low contrast defects – is time consuming with frame cameras.
A single, 16,000 pixel line scan camera is a much better solution. One camera now spans across the material and there are no defective pixels. The rotary encoder output triggers the acquisition of each line of the image. Line exposure time is short to avoid motion blur, but now the illumination can be focused into a line, rather than being spread out over a frame, to get an acceptable image. A Time Delay and Integration (TDI) line scan camera has rows of pixels that integrate exposures in synchrony with the material movement, thus multiplying the effective exposure time by the number of rows of pixels. Last, the gain and offset of each pixel can be adjusted to give a uniform response over the line of pixels, and these adjustments are done in the line scan camera’s hardware.
When to Use Line Scan Technology
Line scan technology is ideally suited to applications in which large, high-resolution, or high-speed image capture is needed, such as in a continuous web application—paper, textiles, metal, or glass tape, for example—or for railway and road inspection, or for satellite imaging or surface topography scanning. Line scan technology works well with free-falling products, including molten glass and steel or postal sorting. Line scan is required for tasks such as inspection of silicon wafers, flat panel displays, solar cells, printed circuit boards, medical diagnostics and other tasks that require large, defectfree images.
Line scan cameras are better than frame cameras when the product speed is high, because you can compensate for short exposure times by using concentrated illumination and Time Delay and Integration (TDI) to increase the photons “harvested” by the camera. A line scan camera does gain and offset correction in the camera, while an area scan camera uses the vision system’s CPU to compute this correction and so is much slower.
How Line Scan Images are Formed
A line scan sensor has one or more lines of pixel sensors. During the camera’s exposure time each pixel accumulates photoelectric charges proportional to the light from the object imaged onto that pixel. At the end of the exposure time the charges in an entire row of pixels are transferred into a readout register. The readout register shifts out the pixel charges and they are amplified, corrected, and digitized to produce the camera output. The readout register shifting is done while the next row of pixels is being exposed. The maximum rate at which exposure and readout can occur is called the “line rate” and is specified in kilohertz (kHz).
To “freeze” the motion of fast moving objects, high line rates are needed. Teledyne DALSA line scan cameras have line rates of up to 200kHz, or 5 microseconds per line of pixels imaged. Multiple “taps” – readout points along the readout register – are used to increase readout speed.
Line Scan Acquisition Interfaces
The pixel data from the camera are transmitted to a vision processor or to a frame grabber for processing. Teledyne DALSA provides cameras with three types of data transmission interfaces.
Gigabit Ethernet (GigE) can be used for data rates up to about 80 Megabytes per second (MB/s). A standard GigE interface receives the camera data at the vision processor. This interface is used on Teledyne DALSA’s GEVA (GigE Vision Appliance) vision processors.
CameraLink interfaces use a frame grabber to receive the data and have one of three formats. The Base format transmits 255MB/s using a single cable. The Medium format is 510MB/s and the Full format transmits 680MB/s, using two cables.
The HSLink interface, pioneered by Teledyne DALSA, transmits 6000MB/s and requires a frame grabber to receive the data. For example, the Piranha3 16k (16,384 pixels in a line) camera uses HSLink to acquire at a 72kHz line rate, about 1,179,000,000 pixels per second.
In the frame grabber or vision processor, the transmitted lines of pixel values are accumulated into a frame – an image with the width (X dimension) of the line scan sensor and a userspecified height, usually no more than 4,000 lines. Frames are processed to perform the machine vision task, as examples, finding defects or reading text (OCR). Successive frames are overlapped in the vision processor, for example to detect a spot defect that spans the top edge of one frame and the bottom edge of the next frame.
Time Delay and Integration
Time Delay and Integration (TDI) line scan cameras have 2 to 256 rows of pixels, arranged vertically. Photoelectrons from each pixel in each row are summed into the row of pixels “below”, in the direction of object motion. The summing is done by shifting the accumulated photoelectron charges.
The shifting and summing is driven by the object’s movement rate, usually signaled by pulses from a motion encoder. TDI cameras effectively multiply the exposure time by the number of rows of pixels and so can provide high-contrast images even when exposure time is short. Another way to look at this is that lines of pixels are electronically moved in synchrony with the movement of the object being imaged and so the exposure time is increased without causing additional blurring due to the object’s motion.
Note that the camera’s exposure time is independent of the line rate except that the exposure time should be less than the inverse of the line rate (seconds per line).
Colour Line Scan Cameras
Colour line scan cameras have rows of sensor pixels with different colour filters or different colour filters on pixels along the row, to detect light of different wavelengths. Typically filters are what we perceive as red, green, and blue (RGB), but some applications, such as satellite remote sensing, use more and different filter types.
Trilinear sensor pixels with Red, Green, Blue filters per line
Bilinear sensor pixels with Green and alternating Red and Blue filters
Colour line scan cameras are particularly useful in imaging printed material. For this task, use a “trilinear” camera with one filter type per line, so that each image pixel has a full set of RGB measures. The different lines of colour sensor pixels are delayed in time, so they all view the same part of the object being imaged.
Lenses for colour line scan cameras must be colour corrected so they do not have significant chromatic aberration. Chromatic aberration occurs when a lens doesn’t focus light of different wavelengths onto the same focal plane, and appears as colour fringes around intensity edges.
If the camera views an object at an angle off of the normal (the perpendicular to the object’s surface), there can be some perspective distortion. Perspective distortion is proportional to 1/distance to the object, and so is slightly different for the different colour rows of pixels. So each row of pixels has a slightly different size and shape of pixel as projected on the object’s surface, and this can cause errors in high-accuracy colour measurements. Make sure the camera is perpendicular to the surface of the object – a good idea even if you are not using colour.
Lighting for Line Scan Cameras
You need light to see objects, and lighting and optics act as “optical processors” to enhance an object’s interesting features. Line scan cameras typically use a “line light”, focused light from a line of LEDs, to illuminate the object along the line of object pixels being viewed. Line lights give high intensity illumination needed for short exposure times (fast camera line rates). Line lights can be “butted” together to give very long lights for viewing wide objects.
A high-powered line light from Metaphase Technologies, Inc.
Line lights are usually positioned behind the object being imaged to give “back lighting” or at a “high angle” above the object, so “looking down” on the object. Back lighting is used, for example, to detect pinhole defects in opaque materials or particle defects in transparent material. High angle illumination is used, for example, to get object colours or outlines or to detect surface defects.
The light intensity depends on the LEDs used, power flowing through the light, age and quality of the lamps, and the efficiency of the lens. These variables make it difficult to specify a light with just the right amount of intensity so, to be safe, we often specify more illumination intensity than we might need.
How to Select the Right Line Scan Camera for Your Application
Three important considerations when choosing a line scan camera are: sensitivity, size (in sensor pixels), and line rate.
Sensitivity
“Sensitivity” asks, “Is the camera getting enough photons to perform the machine vision task?” It is usually difficult to answer this question only from component specifications, as there are too many variables. In practice we estimate the light intensity needed, specify more light intensity than our estimate, and then verify that we have the required sensitivity by testing. Here are some of the variables:
First, as mentioned, the light intensity depends on many different variables.
Second, the lens optics reduces the light to the camera. This reduction can be significant for high magnification lenses or when the lens is stopped down to get more depth of field. Third, “sensitivity” depends on the response of the camera – its ability to convert photons to photoelectrons. For increased sensitivity, you can use a camera with larger sensor pixels. Line scan cameras have 100 per cent “fill factor”, the percent of the sensor pixel that accumulates photons. Area scan cameras usually have smaller pixels and lower fill factors, and make up for this loss in response by requiring longer exposure times. Finally, a TDI line scan camera can be used to increase exposure time by electronically moving the camera’s sensor lines with the object.
Size (in sensor pixels)
To determine camera size (in sensor pixels) you need to specify field of view (FOV) and minimum defect size. The camera should have enough resolution to have at least 3 or 4 pixels “covering” the minimum defect size. If, for example, the FOV is 12” and the minimum defect size is 0.005”, then:
(FOV/minimum defect size) x (3 pixels coverage)
(12/0.005) x 3 = 7200 pixels
You could use an 8K (8,192 pixel) line scan camera, or two 4K (4,096 pixel) line scan cameras side-by-side with some overlap in their horizontal fields of view.
Line Rate
The line rate is set by the FOV, part speed, and object pixel size. For example, if the FOV is 12”, the part speed is 60” per second, and you are using an 8K (8,192 sensor pixels) camera, then:
(Object pixel size in the FOV) = FOV/(camera size in sensor
pixels) =
12/8192 = 0.001465” object pixel size in the FOV
Line rate needed: 60”/0.001465” = 40.956kHz
You might use a Teledyne DALSA Piranha4 70kHz camera.
Optics for Line Scan Cameras
The lens gathers light from the object to improve sensitivity and magnifies (or minifies) the FOV to match the size of the camera’s line sensor.
You can compute the magnification as:
Magnification = (camera pixel size in microns)/(FOV object
pixel size)
In the above example, using the Piranha4 camera, this would be:
Magnification = (7.04 microns) / (37.211 = 0.001465 inches
converted to microns) = 0.189
So a reduction in size of 1/0.188 = 5.286 from the “real world” FOV to the camera’s line sensor. This ratio (5.286 in this example) is the “inverse magnification”.
Next, you must specify a “working distance”, the distance from the camera’s face plate (the front, flat surface of the camera) to the object being imaged (for example, the web material). From this you can get the focal length of the lens required:
Focal Length = (working distance) / (inverse magnification + 1)
In this example, a working distance of 314.28mm = 12.37 inches is chosen to give a focal length of 50mm, which is a standard focal length for lenses. The working distance is usually adjusted to match a standard lens focal length.
Besides the focal length, you will need to know the lens mounting type. Smaller line scan cameras, up to 2,048 pixels in some cases, can use a C-mount lens with a 1 inch aperture. For longer line scan sensors an F-Mount, M42 or M72 mount lens is required.
Line Scan Camera Synchronization
Line scan camera exposures must be synchronized to the object’s movement. This is usually done with an encoder that outputs a pulse for each specified amount of object movement. After some number of encoder pulses, the line scan camera is triggered to take a line image.
We usually want “square pixels”, that is pixels of equal width and height in the field of view. To get square pixels, the camera must be triggered every time the object moves a distance equal to the object’s pixel size. In the above example, a pixel size in the field of view is 0.001465 inches so, for square pixels, the camera has to be triggered every time the object moves that distance.
To set the line acquisition trigger distance, you can (1) select or program the encoder’s pulses per distance or revolution, (2) select the mechanical ratio between the encoder and object movement, and (3) program the number of encoder pulses between camera line acquisitions.
Using the above example, suppose that the object is on a conveyer belt and a rotary encoder is connected to a conveyer roller of known circumference. For square pixels, we want a camera trigger pulse every 0.001456 inches of conveyer belt movement. We assume no mechanical slippage between the object movement and the rotation of the encoder.
We pick an encoder with 4,096 pulses per revolution and the conveyer roller’s circumference is 3 inches (so diameter = circumference / pi = about 1 inch). The distance per encoder pulse is:
Distance per pulse = (roller inches per revolution) / (encoder
pulses per revolution)
= 3 inches / 4096 pulses = 0.0007324 inches per pulse
This is too small a distance by about a factor of 2. If the roller circumference is fixed, you could use a PLC’s “axis” inputs or the encoder inputs on a Teledyne DALSA Xcelera frame grabber to “divide” (really, count-down) the encoder pulses by a factor of 2. Or you could pick an encoder with 2,048 pulses per revolution. Either would give:
Distance per pulse = 2 * 0.007324 = 0.0014648 inches
Pixel Correction with Line Scan Cameras
An advantage of a line scan camera is hardware correction of individual pixels’ responses.
There are many sources of variation in pixel responses. Lighting is never completely uniform – You typically see some variation due to variations in brightness of individual lamps in the light and some fall off in intensity towards the edges of the light. Most lenses attenuate light away from the optical axis (the centre line through the lens). Individual pixels in the camera have some variation in their offset and gain due to variations in the camera manufacturing process.
Each pixel’s offset and gain is adjusted to compensate for these sources of variation. This could be done by exposing all pixels to the lowest intensity on the object and the highest (but not saturating) intensity on the object. Offset correction subtracts the lowest intensity value. Gain correction sets the range between the (corrected) lowest intensity value and the highest intensity value to be constant across sensor pixels.
Pixel response before gain and offset correction
Pixel response after gain and offset correction
Line Scan Cameras at Work
Here are two “real world” applications of line scan cameras.
Multiple Camera Web Inspection
Line scan technology is the method of choice in continuous web applications. As an example we will look at a production line for plastic film that is 12 feet wide and moves at 350 linear feet per minute. We are looking for small defects, such as holes, dirt or contaminants. Attempting this level of inspection with an area scan camera is very challenging due to the wide area and high speed of the web. Doing this by human vision inspection is impossible.
Defects are classified based on size range, such as from 50 to 100 microns or 200 microns, and contrast type. A bright object might be a hole and a dark object might be a contaminant. For this level of detail and web width, six Gigabit Ethernet 2,048-pixel line scan cameras are needed. Backlight is used, so a line light shining through the web and into the camera:
The aggregate data rate is 73 million pixels per second.
An image for processing is 3560 lines.
With a continuous web application, there is no way to immediately reject a defect; the web is moving too fast to stop the process. Instead, a “roll map” records and describes the position and type of defects. This screen shot shows the portion of the roll map made by camera 5:
This shows a real-time view (on the left). The roll map, on the right, the y axis identifies linear footage (in roller revolutions) and the x axis shows the defect position. When the product roll is sent for finishing, defects can be picked out as the product is unwound.
Colour Line Scan
A colour line scan camera is used for inspecting medical trays that incorporate rivets, grommets, and brackets of various colours. These elements could not be easily distinguished and inspected using gray-scale (intensity) images.
Operator’s view of the medical tray; a missing part indicated by a thick, red box (upper, right)
The manufacturer produces trays of many different sizes, the largest of which is 27 inches in length and 18 inches in width. A 2K colour line scan camera is mapped to 20 inches on the horizontal to accommodate the largest medical tray; with line scan technology, there is no limit on the vertical size. Two 24 inch line lights illuminate the entire tray, which is moved along a conveyor. An encoder on the conveyor is synchronized so that the colour line scan camera triggers as the tray comes into the field of view.
Summary
Line scan technology is used in applications that image large objects, are high-resolution, are high-speed, need perfect images, or need real-time correction of pixels’ responses. The camera has a single line of pixels and either the camera or object is moved perpendicular to that line of pixels to build up a two-dimensional image of the object.
To get high brightness, “line lights” are focused where the line scan camera is viewing the object. Time delay and integration (TDI) cameras give longer exposure times by electronically moving lines of pixels in synchrony with the motion of the object.
Applications where line scan cameras are the best choice include inspecting web materials, as in plastic or paper manufacturing, inspecting continuous “objects” such as road and rail inspection, or where high-quality images are required as in printing inspection.
When implementing line scan technology, care must be taken to synchronize the camera acquisition to the object’s movement. Starting costs may be higher than for frame cameras, but line scan technology solves problems that frame cameras cannot easily solve or solve at all.
