Efficient Network Architecture and Training Method for Unspecific Real Noise Reduction

Abstract

Image denoising is a process that generates a high quality image from a low quality image which is degraded by external noise such as additive white Gaussian noise (AWGN), speckle noise, and impulse noise. Image denoising is a major research area in image processing research field because of its wide range of use such as medical image denoising, satellite image denoising, and compression noise denoising. Among many uses, object detection and recognition in autonomous vehicles significantly increased the attention of researchers on image denoising as it is essential to remove noise from the image to improve the performance of object recognition. Owing to these demands on image denoising research, numerous image denoising solutions have been proposed. However, there were limited improvement in image denoising performance prior to the application of deep convolutional neural network (CNN). Recently, numerous image processing studies including denoising have applied CNN and exhibited drastically improved performances. However, the conventional denoising solutions suffer from serious limitations. First, as they are trained for specific type of noise with certain noise level, they show limited performance when they are applied to real world systems. As it is impossible to acquire the type and intensity of the noise from noisy images, the conventional denoising solutions that require the information of the noise as an input data cannot demonstrate their performance. Moreover, even if they acquire the characteristics of the noise, it is inefficient for real world systems to apply the conventional denoising solutions because they require respective set of trained parameters for each noise level. This requires not only the significant training time but also the considerable amount of memory space. In this dissertation, a densely connected hierarchical image denoising network (DHDN) is proposed which exceeds the performances of the state-of-the-art image denoising solutions while overcoming the limitations. Image denoising performance of DHDN is improved by applying the hierarchical architecture of the modified U-Net; this enables the network to use a larger number of parameters than other methods. In addition, DHDN shows a competitive computational complexity despite its numerous parameters thanks to the model architecture that compensates the computational complexity from the parameters. Finally, model ensemble and self-ensemble methods are applied to DHDN proving that these methods can improve the objective and subjective performance of denoising networks. On the other hand, DHDN is trained to handle a wide range of Gaussian noise level with a single set of trained parameters. Furthermore, DHDN is trained to handle unspecified real world noise without any information of noise characteristics with a single set of trained parameters. It enables DHDN to completely overcome the limitations of the conventional denoising method. Experimental results demonstrate that the performance of DHDN is outperforming the performance of the conventional denoising solutions in reducing the AWGN; this is notwithstanding the fact that DHDN is trained to handle a wide range of noise levels with a single set of trained parameters. The performance of the network in reducing unspecified real noise is also validated by winning the second place in NTRIE 2019 real image denoising challenge sRGB track and the third place in the raw-RGB track. Further analysis demonstrates that the architecture of DHDN is more efficient than the models of other challenge participants even with the large number of trainable parameters and competitive performance.