Multi Label Image Classification using Dual Aggregated Neural Network

Abstract

While many deep convolutional neural networks show promising performance in various classification tasks, many objects appearing in very different sizes, shapes, and appearances cause difficulties in multi-label image classification. This thesis introduces a dual aggregated network on pyramidal convolutional features for multi-label classification. The proposed method includes feature-level and classifier-level aggregation to learn discriminant information of various target objects in the image. First, the feature-level aggregation collects the convolutional activation maps from the multi-scale pyramid network, and then it pools using various pooling windows for generating localized features of each object. The feature aggregation method is elaborately designed so that the responses from the objects with different sizes, aspect ratios, and shapes are properly reflected in the aggregated activation map. Unlike conventional methods, this process does not require the region proposal step, which reduces the computational cost significantly. Second, this thesis introduces the classifier level aggregation for combining classifiers of each object class. To maximize the discrimination power of each class, the proposed method is trained one-vs-all classifiers for individual classes using the class-wise loss function. For each test image, the scores from the class-wise classifiers are aggregated to get the final multi-label classification result. By integrating the above two feature and classifier level aggregation methods, the proposed network can be trained in an end-to-end fashion, which is not possible for the conventional multi-label classification algorithms using region proposals. Extensive evaluations on PASCAL VOC 2007, PASCAL VOC 2012, and WIDER-Attribute shows significant performance improvement compared to existing methods.