Human Intention Recognition through the HoloLens for Human-Robot Interaction

Abstract

Human Intention Recognition through the HoloLens for Human-Robot Interaction Taeyhang Lim Department of Interdisciplinary Robot Engineering Systems The Graduate School Hanyang University The goal of this paper is to revolutionize human-robot interactions by enabling humans to intuitively control robots through their gaze. To achieve this, a head-mounted device called HoloLens 2 was used to capture the user’s gaze, offering a natural and intuitive means of robot operation. Previous studies have outlined the challenges of using traditional robot control methods as they require manual handling of control equipment and the challenge of the Midas Problem of distinguishing between goal and non-goal gaze. To address the challenges of using gaze for robot control, our research is divided into two segments: (1) Hands-Free Robot Control: A hands-free technique using HoloLens 2 to control the Robotnik Summit- XL mobile robot through eye gaze through an augmented reality (AR) environment. Stable robot navigation is achieved using admittance control based on gaze positioning. In a comparative usability study, we evaluate joystick control against our proposed method, with System Usability Scale (SUS) and Single Ease Question (SEQ) assessments indicating adept user collaboration with the robot. This highlights HoloLens 2 as a potential alternative to joysticks for hands-free robot operations, enhancing human- robot collaborative efficiency when users’ hands are occupied. (2) Gaze Intention Prediction: A method to distinguish between goal-directed and non-goal-directed gaze. This was done by creating an Area of Interest (AOI) on each object based on goal-directed gaze patterns as people gaze at different locations on a single object with different intentions. Through the performance comparisons between methods with and without AOIs focus on pick-and-place robot, the use of AOIs yields a 19% improvement in the F1 score compared to the method without AOIs. These findings strongly suggest that predefined AOIs enhance gaze intention prediction, allowing human-robot collaboration that can significantly enhance efficiency and productivity. These two proposals collectively demonstrate the potential of gaze-based control for hands-free robot operation and the effectiveness of AOIs in improving gaze intention predictions. This underscores the promise of gaze as a natural and intuitive means of human control, with the potential to enhance the efficiency of human-robot collaboration across various applications.