Simulation and Automation for Tower Crane 3D Lift Planning Using Reinforcement Learning

Abstract

Tower crane lift planning plays an important role in affecting productivity by providing efficient lift schemes for moving resources. Automizing efficient lift planning from a large number of potential candidates is a complex optimization problem. To address this issue, researchers have developed mathematical models combined with various path-finding algorithms. However, most previous studies were only limited in grid trajectory from coordinates rather than motions based on the mechanic performance of the tower crane. Another problem is the difficulty in estimating the task time of the planned path. These are important when liftings are executed according to the plan. To address those challenges, this study proposes the reinforcement learning-based method for tower crane motion planning that could conduct a spatio-temporal lifting plan which is based on the actuator system of the tower crane. As the method for tower crane motion planning, representatives on-policy and off-policy algorithms with various training strategies are proposed and tested. Among which a on-policy RL-based method presents realistic motion planning results that can be feasibly executed by crane operators. The results show successful learning on lift task procedures, stable and practical lifting plans with a failure ratio of 3%, and a coordination ratio of 28% with a positive evaluation of expert operators. The task time estimated from planned motion shows a significant correlation ratio of 0.6857 and an average error of 19.3 seconds with the actual lifting time. These results indicated that the proposed method is significantly adaptable to the actual sites. The proposed approach is promising for planning feasible lifting paths and estimating reasonable lifting times, which help generate and review lifting plans given the site conditions.