Interactive Pixel-unit AR Lip Makeup System Using RGB Camera

Abstract

In this paper, we propose an AR (Augmented Reality) lip makeup using bare hands interactively using an RGB camera. Unlike previous interactive makeup studies, this interactive lip makeup system is based on an RGB camera. Also, the system controls the makeup area in pixels, not in polygon-units. For pixel-unit controlling, the system also proposed a ‘Rendering Map’ that can store the relative position of the touched hand relative to the lip landmarks. With the map, the part to be changed in color can be specified in the current frame. And the lip color of the corresponding area is adjusted, even if the movement of the face changes in the next frame. Through user experiments, we compare quantitatively and qualitatively our makeup method with the conventional polygon-unit method. Experimental results demonstrate that the proposed method enhances the quality of makeup with a little sacrifice of computational complexity. It is confirmed that natural makeup similar to the actual lip makeup is possible by dividing the lip area into more detailed areas. Furthermore, the method can be applied to make the face makeup of other areas more realistic.