인간 시각 특성에 기반한 비간섭 증강 현실

Abstract

Over the last decade, augmented reality (AR)’s growth and progress have been remarkable and it has been applied to a number of fields such as medical, education, entertainment, and so on. Until recently, most AR prototypes concentrated on how well to register virtual information with the real world (technical factors) but did not significantly concern themselves with how much potential users would be convenient with using these systems (human factors). This is the motivation of the dissertation and the human factors will be mainly discussed here. Specifically, this dissertation discusses the user intrusiveness of visual markers or patterns which are necessary for accurate registration between the virtual information and real world and how to make them nonintrusive. Simply, not using markers or patterns may be a solution for AR to be nonintrusive. In this regard, some marker-less methods and hybrid methods combined with sensor-based methods (although sensor-based methods are not the concern of this dissertation) have been proposed. However, it is well-known that the performance of those methods is worse than that of marker-based methods and they usually need user intervention, which would bother users. Moreover, in projection-based AR, use of markers or patterns is indispensable because it is practically impossible to estimating the scene geometry or radiometry without markers or patterns. In this dissertation, it is basically aimed for using markers or patterns and tried to make the markers or patterns visually imperceptible. To discriminate the related technique or technology from the existing AR techniques or technologies, the term of nonintrusive AR is defined. It is well known in the field of computer vision and graphics that the intensity of all irradiance observed at every point in the 3-D scene, coming from every direction is described with a single function, called plenoptic function. Therefore, one can come up with hiding visual markers or patterns by transforming the plenoptic function. Then, the principle of all nonintrusive AR methods can be explained as modulating a single or multiple parameters of plenoptic function. It indicates that the performance of nonintrusive AR methods is determined as which parameter is modulated and how it is modulated. This dissertation analyzes the previous nonintrusive AR methods based on this paradigm and proposes more efficient nonintrusive AR methods. Additionally, all the nonintrusive AR methods were originated from utilizing different characteristics of the human vision system to make markers or patterns visually nonintrusive. This is also related to the plenoptic parameters they modulated. In this regard, this dissertation proposes a taxonomy for nonintrusive AR methods. For the sake of convenience, AR is classified into two categories: 2-D AR, 3-D AR. Desktop AR, wearable AR, mobile AR, and so on belong to 2-D AR because they use a 2-D display such as monitor, HMD, PDA, cellular phone, and so on to show AR images. Projection-based AR belongs to 3-D AR because it displays (projects) AR images on 3-D real scene. Different frameworks for 2-D and 3-D AR to be nonintrusive are proposed and evaluated in the aspect of accuracy and usability. First in 2-D AR, the previous nonintrusive methods that concern with modulating different plenoptic parameters are reviewed by analyzing their pros and cons. Then, the wavelength parameter of plenoptic function is carefully modulated (to propose efficient nonintrusive AR) in this dissertation. Specifically, IR markers are used which are visually imperceptible to human eyes and an effective method of tracking IR marker is proposed. The usefulness of IR marker is thoroughly discussed through experiments, subjective evaluation, and comparison with other types of visually imperceptible markers. The usefulness of UV marker which is another type of visually imperceptible marker is additionally discussed. Next in 3-D AR, the previous nonintrusive methods that concern with modulating different plenoptic parameters for making patterns imperceptible are reviewed by analyzing their pros and cons. Then, modulating the time parameter of plenoptic function is carefully examined in this dissertation. Specifically, complementary patterns are used which are visually imperceptible to human eyes if they are projected alternatively at a high speed. Detailed discussion of the perceptibility of complementary patterns is given in the end of the dissertation. As a matter of fact, complementary patterns have been used in other research fields namely 3-D reconstruction. The novelty of my research lies in the fact that complementary patterns are embedded into AR images and the embedded patterns are detected (extracted) from the projected AR images. Several methods of embedding complementary patterns and detecting the embedded patterns are proposed and their performance is verified through experiments. In general, projecting the complementary patterns less than 75Hz (conventional projectors work at 60Hz) may cause flickering (affecting the pattern visibility). This flickering is proportional to the intensity of complementary patterns and also depends on the color distribution of AR images. To alleviate the flickering, a content-adaptive method of embedding complementary patterns with locally different strength and into locally different channel based on the AR contents is proposed. A content-adaptive method for determining the block size by analyzing the AR contents is additionally proposed. It is demonstrated that the content-adaptive methods can significantly reduce the intrusiveness of patterns and produce better compensation results by comparing it with the previous non-adaptive approaches through a variety of experiments and subjective evaluation.