3D Dense Reconstruction from Optical Flow of a Monocular Video Sequence

Abstract

Recently, innovative technologies that perceive environmental information, such as autonomous driving and AR/VR, have shown potential, and the emphasis on accurate and dense perception of the surrounding environment is increasing. However, traditional feature-based mapping algorithms use sparse structural information, which limits the generation of high-resolution maps. In this thesis, we propose a novel method for reconstructing a precise reconstruction of dense maps using an optical flow estimation network that exhibits enhanced performance with the recent development of deep-learning. Whereas scene geometry understanding tasks have concerns as to whether ambiguity is increased by interaction, the proposed method integrates mathematical methodologies by utilizing pose information from the conventional SLAM algorithm and estimating optical flow using the Recurrent All-Pairs Field Transforms (RAFT) network by \etal{Teed}~\cite{teed2020raft}. The global dense map generation algorithm ensures precision by deducing optical flow between the frames by the learning-based method, and reconstructs 3D points for all pixels by multi-view triangulation using estimated optical flow as input. Subsequently, the outlier rejection step eliminates incorrectly estimated pixels to increase the accuracy of the final reconstruction. Additionally, a mask is generated by calculating the parallax. Finally, in the process of accumulating frames, a 3D mesh volume is created to restore a high-visibility precision map. As a result, the proposed dense reconstruction method can estimate the depth value on a metric-scale and generates dense maps with high visibility. Experimental results on KITTI odometry dataset, ETH Multi-FoV dataset, and EuRoC MAV dataset demonstrate integrated dense map on diverse situations.