High-Performance Deinterlacing Techniques Preserving Edge Details

Abstract

The ultimate purpose of deinterlacing is to convert interlaced scan format to progressive scan format for digital TV receivers, DVD players, and etc. Deinterlacing can be interpreted as a kind of post-processing that is the last step to improve the perceptual quality of the video obtained from the decoder. Basically, deinterlacing encompasses image (line) interpolation to obtain high resolution image from low resolution image. A myriad of deinterlacing methods have been developed via various approaches from linear techniques to non-linear content/motion adaptive techniques. However, they have some drawbacks such as blurring, line flicker, serration, and line crawl. Although there are methods that successfully estimate the missing lines, they are not suitable for real-time applications because they require too much time to complete the overall process. Therefore, we propose the three different approaches that reduce the motion artifacts and preserve the continuity of the edge, while restricting the required CPU processing time to make them available for consumer electronics directly. The high-definition television (HDTV) broadcasting system, such as ATSC, in the US, Japan, and Korea adopts an interlaced scanning format (1080i, 1080�e1920 resolution with only 540 lines scanned in each frame). The concept of interlaced scan was thought of in the first place due to a well-known fact that the human visual system (HVS) is more sensitive to flicker in large areas. Interlacing artifacts, such as line flicker, serration, and line crawl, annoy the HVS, especially, when the screens get bigger and brighter, and the frame rate get higher. Indeed, the size and brightness of the progressive displays, such as PC monitors, liquid crystal displays (LCD), and plasma display panels (PDP), are getting bigger and brighter currently, and they are dominating the world market. Hence, in order to guarantee compatibility with existing TV broadcasting standards and eliminate the interlacing artifacts, there is (and will be) a need for conversion between the interlaced and progressive scanning formats. The deinterlacing methods can be categorized into non-motion compensated (Non-MC) method and motion compensated (MC) method. MC methods propagate information along the motion trajectory, which transports detailed video content existing in neighboring fields into the lines in the present field where they are missing. In contrast, instead of using motion compensation, non-MC methods utilize spatial, temporal, or spatio-temporal information based on motion activity and edge direction. Since the proposed methods focus on providing a high image quality with low-complexity, all the proposed methods belong to non-MC method because motion estimation and compensation procedures in MC methods accompany a large number of operations and may produce unreliable motion vector. Three different approaches reducing the computational complexity of Wiener filtering-based deinterlacing (WFD) are presented in this dissertation because WFD has the complexity of O(n3) where n(=8) denotes the number of columns in the data matrix. The first proposed approach is the computation-aware algorithm selection which assigns the optimal deinterlacing method to the most proper region to reduce the hit rate of WFD. The second approach utilizes the direction-oriented inverse-free Wiener filtering to reduce the neighbor set, i.e., n is reduced to 4. The third approach starts from generalizing the direction filter to reduce the dimension of the weight vector, i.e., n is reduced to 3. To avoid from extracting invalid local edge direction, the local surface model-based deinterlacing method is proposed. The proposed method uses the local surface model which is modeled via quadratic equation having two-dimensional coordinate variables. Since this method only requires linear operations, its complexity is O(n) which is the same as the conventional spatial deinterlacers, where n indicates the utilized neighbor set. Finally, the proposed spatial deinterlacing methods are applied to the motion adaptive deinterlacing to develop the motion adaptive deinterlacing methods that are robust to the motion artifacts and preserve the edge details. The validity of the proposed methods is proved that they are robust to the fast motions and complex regions. Also, together with the temporal methods used in static regions, significantly enhanced video quality can be obtained. Several spatial deinterlacing methods including the simplified WFDs and LSM are presented in this dissertation. Since they are robust to the motion artifacts and preserve edge details, they are applied to the MA deinterlacing framework to play a key role in the moving regions classified by the motion detector. Based on the star graph comparison with the nine representative test sequences, the proposed MA deinterlacers provide better visual performance than conventional deinterlacing schemes regardless of test sequences. In particular, the proposed scheme significantly exhibits better performance in sequences with both of motion and edges.