Deep Learning based Dynamic Object Classification using LiDAR Point Cloud Augmented by Layer-based Accumulation for Intelligent Vehicles

Abstract

The intelligent vehicle must identify the exact position and class of surrounding objects in various situations and consider the interaction with them. For example, an intelligent vehicle should be aware of a bike suddenly popped out, and it should stop to prevent a collision or plan a safe driving route to avoid pedestrians in the middle of the night based on recognized information. For this reason, the light detection and range sensor called LiDAR is widely used in intelligent vehicles, because this sensor provides information in the form of a point cloud that can be used to localize and classify surrounding objects. However, while vision-based object detection and classification systems made remarkable progress with the introduction of the deep learning, LiDAR-based recognition system cannot provide sufficient classification performance even with the deep learning technologies. The main reason is that the LiDAR point cloud does not have enough shape information to classify the dynamic object due to the sparsity of the points. To address this problem and close the gap with the vision-based system, we propose a method to enhance the deep learning-based classification performance by augmenting shape information of the LiDAR point cloud. In order to enhance this shape information, we propose a layer-based accumulation algorithm based on the three degree-of-freedom motion of a dynamic object and three-dimensional information for reconstructing the shape of the object effectively. In order to classify the LiDAR point cloud based on augmented shape information, we exploit a deep neural network which is not influenced by point-order and uses a point cloud as a direct input. To effectively train and test this model with shape information, we generate the dataset using the three-dimensional computer-aided design model without tedious efforts for labeling. In the experimental results, the proposed accumulation method outperforms existing registration-based methods. Moreover, the classification performance of the LiDAR point cloud with augmented shape information is superior to the classification performance using the point cloud without accumulation in the real-vehicle test.