자연스러운 인간 컴퓨터 상호작용을 위한 하이브리드 비전 시스템

Abstract

Recently, human computer interaction (HCI), which has been widely used in various natural interactive interface systems, has been designed by vision-based systems. In a vision-based system, the most challenging tasks are to allow natural and accurate interactions. The various methods for natural interactions have been proposed using the depth-camera based interaction system [1]. However, the system is able to interact with a small number of movements of the users. Some of the existing methods interact within constrained ranges of motion, and the accuracy of the interactions is just guaranteed in the controlled environments. Thus, most of the depth-based HCI methods are focused on detecting or tracking the specific region-of-interest. The systems also are not suitable for the fine motions of users. Unfortunately, camera vision-based systems have not been good solution for HCI. The HCI is difficult to apply to fully vision-based systems because the accuracy of interaction is rapidly decreased according to the degree of the user’s movement or the surrounding environment. In this dissertation, we propose two hybrid vision-based systems: a heterogeneous vision system for robustly human region detection and ego-exo vision-based system for accurately motion estimation. The natural-interaction systems are implemented by two vision-based methods using color and infrared cameras, using ego-exo cameras. Most of HCI systems employ the face and hand regions, but similar regions and cluttered environments can interfere with these region-of-interests. Thus, the robust region detection is very important for interaction using motions of region-of-interests. For this purpose, we propose a viewpoint-coincident camera system, which can capture color and infrared images at the same time, and then apply the images to detect the region-of-interests robustly. The hybrid method using color and infrared images simultaneously detects the region-of-interests such as human or marker regions, which are used for natural interaction. Next, we propose a novel interaction method that is designed for user’s fine motions, using ego-exo cameras. In the case of fine motions (e.g., breath-induced motion or wrist motion for the purpose of medical diagnosis or rehabilitation games), most vision-based systems are not suitable to ensure the accuracy of motion estimation. For solving this problem, we propose a hybrid motion estimation method using ego-exo cameras. In the vision-based interaction method, the system gradually reduces the region of interest for natural interaction, but the accuracy of motion estimation is essential. In the region-based interaction methods, the user’s specific regions or body parts are defined as region-of-interests for interacting with human interface. Therefore, skin or specific color regions such as the face or hand regions are detected by a new method, and then head motions or hand gestures are used for interaction. Moreover, the feature-based method can interact with fine motion by estimating accurate motions from feature tracking. However, the method is more sensitive to small errors. The vision-based system is important to overcome that sensitivity through accurate fine motion estimation. Consequently, for natural HCI, the vision-based system has to detect the interaction regions robustly, and estimate fine motion with high accuracy. In this dissertation, we have tried to solve the above problems with two hybrid interaction methods. In addition, practical applications to provide the natural interaction have been implemented. The viewpoint coincident camera system is able to robustly interact with users, and the ego-exo camera-based hybrid vision system improves the accuracy of fine motion estimation by reducing rotation error into 12% or less in comparison to the existing single camera method. It is expected that proposed methods can provide a practical solution for robust and accurate natural HCI.