Upgrade image system performance without changing optics – forget pixel size

For many systems the optical format was decided by the initial choice of the image sensor, for instance a 1/3” VGA sensor or 2/3” HD CCD image sensors with pixel sizes of 6 or 7μm. These legacy systems are now due for upgrades.  We all would like to take advantage of the sensitivity, lower read noise and higher pixel throughput of the latest CMOS image sensors. However, trends with CMOS image sensors also mean much smaller pixel sizes which can result in a much bigger overhaul of the system than is desired.  Read that as a new optics design.

Adimec’s proprietary Adaptive Resolution™ provides the option to emulate the original optical format using programmable digital image down-scaling. Adaptive Resolution was developed so that any new camera fits on existing optics and on the existing interface/displays; for example, using a state-of-the-art Quad HD CMOS image sensor to replace an older VGA CCDs with larger pixels on VGA displays. This blog discusses the performance of Adaptive Resolution to be able to evaluate this new technology for a legacy system with a certain optical format and output image size in terms of pixels.

Why should I use Adaptive Resolution?

Image sensor technology is rapidly moving from CCD to CMOS technology even for industrial and security applications. This is driven by the CMOS process developments for consumer electronics. The latest generations of CMOS image sensors are outperforming state-of-the-art CCD image sensors in all aspects:

• Lower read noise
• Larger dynamic range
• Better handling of local over-exposure (smear, blooming)
• Lower power
• Higher speed, allowing to go to UHDTV resolutions

Furthermore, CMOS process technology improvements have allowed for shrinking of pixels while preserving important imaging parameters like: QE, charge handling capacity per square μm, and more.

The advantages of smaller pixels are:

• Lower Size, Weight and Power (SWaP)@ the same resolution
• Higher resolution & frame rate @ the same SWaP
• Lower costs

But smaller pixels also raise some concerns:

• Need different/better optics (MTF, mechanical stability)
• Lower dynamic range, though DR at same pixel size is better

Adaptive Resolution exactly tackles these concerns; it allows the system architect or designer to select a new CMOS image sensor without changing optics and losing dynamic range. Advanced interpolation algorithms for cropping and scaling in the camera create virtual pixels of the required size to match the output of the legacy system. This results in the same FOV, combined with a better system performance:

• Higher sensitivity
• Higher dynamic range
• Higher MTF, especially for Bayer and RGB color

This also provides the additional benefit of digital zoom capability.

Adaptive Resolution can be used with a CMOS image sensor with more pixels and with the same or larger optical format than the existing cameras in the system.

How does Adaptive Resolution work?

The image scaling algorithm that is used involves a 2D re-sampling of the intrinsic new image sensor pixel-array, i.e. the image.  The re-sampling process is implemented as a cascade of two successive 1D scale operations (first vertical and then horizontal or vice versa). The 1D resampling operation is performed by means of a poly-phase filter. The filter selected preserves sharpness and limits noise and artifacts. 

A demonstration of Adaptive Resolution

To demonstrate Adaptive Resolution, we compare Adimec’s TMX7-DHD camera using the ICX674 CCD sensor with 4.54 μm square pixel size, with the TMX-50 camera using Sony’s IMX250 CMOS image sensor with 3.45 μm square pixel size using Adaptive Resolution technology to run at a virtual pixel size of 4.54 μm. The grey area in Figure 1 represents the overlap of the physical area of both image sensors. This size (in millimeters) determines the Field of View (FOV). The digital algorithms emulate the 4.54 μm pixel size and operation of the camera as if it were the ICX674 CCD camera. The result is a camera with the same FOV and same number of pixels and pixel size. [Note: in this comparison the physical size of the IMX250 image sensor is slightly smaller than the physical size of the TMX7-DHD image sensor which reduces the resolution to 1874×1080 instead of the full 1920×1080.

Figure 1: Schematic of Adaptive Resolution from Sony IMX674 and IMX250 based cameras.

Figure 2: TMX7-DHD CCD Camera (4.54 um pixels) – TMX50 CMOS (4.54um Adaptive Resolution)

Figure 3. TMX7 -DHD and TMX50 with Adaptive Resolution

The visual results with the same image scene are compared in Figure 2. No difference is readily visible in the images of Figure 2. By zooming in, small sharpness improvements can be detected, such as looking at the “Nieuw recept” lettering in the blue dots of Figure 3.

Thus, we see a graphic demonstration of Adaptive Resolution in action reaping the benefits of CMOS and avoiding the cost of new optics associated with smaller pixel size.