Bayer Pattern Demosaicking Using Local-Spectral-Correlation Based Directional Interpolation

Abstract

Most digital cameras capture images with a single-sensor array, and the surface of this sensor is covered by a color filter array (CFA). Each sensor captures only one of the three color channels such as red (R), green (G), or blue (B), at each pixel position. The captured image, mosaic-like picture, has to be converted to a full-color image by exploiting the originally captured adjacent pixel value, and this process is called demosaicking or CFA interpolation. The Bayer CFA pattern is the most widely used one to provide the CFA image where the G values are sampled on a quincunx grid, while the R and B values are sampled on rectangular grids. The G components are measured at higher sampling rate than other color channels because the peak sensitivity of the human visual system lies in the medium wavelengths, corresponding to the G portion of the spectrum. Due to the characteristic of Bayer CFA pattern, the G channel suffers less from aliasing than the other color channels, and details are well preserved. Therefore, most demosaicking algorithms interpolate the G channel first. To interpolate the missing channel accurately, the correlation between the R, G, and B components is exploited because the three color channels are very likely to have the same texture and edge locations. There are two different correlation models: the constant color difference and the constant color ratio rule. The concept of the constant color difference model assumes that intensity difference between color channels varies slowly and color differences are locally constant. The second model is based on the assumption that the ratios between color channels are similar over small local regions. From many experiments, the color difference model is found to be more efficient than the color ratio model. In addition, color difference model can be implemented with lower complexity and better fits linear interpolation model. Until now, based on color difference rule, many demosaicking algorithms have been proposed to solve various demosaicking problems. Recent researches claim that the existing test set is not suitable for demosaicking performance evaluation. The Kodak image set was widely exploited to evaluate the demosaicking performance, however these images are scanned version of film-based photos, and the characteristic of Kodak dataset is not matched with that of images acquired by digital devices. The new dataset, cropped from high quality images, has been exploited as alternatives, and it is called McMaster dataset. The hue and saturation characteristics of McMaster images are arguably closer to those of these days. Many algorithms focused on Kodak images show severe demosaicking artifacts when these techniques are applied to McMaster dataset. After the new dataset had been announced, several demosaicking algorithms were proposed targeting these images. Although these algorithms improve objective performance, the demosaicking artifact problem still remains. In this dissertation, two different directional interpolation algorithms are proposed focused on the new dataset. The first method, called multi-length directional interpolation filter (MLDIF), uses different length of interpolation filter to generate directional estimates. The filter length is decided based on the similarity of small region to exclude uncorrelated pixel from the reference. A four-direction residual interpolation (FDRI) is secondly proposed algorithm in the dissertation. This algorithm utilizes the characteristic of guided filter which can transfer the structure of the guidance image to the filtering output. The tentative image is generated by using the guided filter, and the difference between original pixel value and tentative pixel value is set as residual. Using the residual value, the four directional estimates are generated, and they are combined using directional weights. The proposed algorithms have been tested on eighteen McMaster images and eighteen Laurent Condat images. The proposed FRDI algorithm provides the best performance in term of color peak signal-to-noise ratio (CPSNR), S-CIE L*a*b* ∆E* score, and FSIMc value. In addition, the proposed algorithms successfully remove the demosaicking artifacts compared to eight reference demosaicking algorithms. The subjective and objective results prove that the proposed FDRI outperforms the others.