Real-time 3D crane workspace update using a hybrid visualization approach

Crane operators often face poor visibility and collision hazards during lifting operations. Dynamic three-dimensional (3D) modeling of the crane workspace helps to alleviate this by identifying potential collision hazards and offering visual assistance and feedback in real time to the operator. This paper proposes a real-time 3D crane modeling and updating framework using a hybrid visualization approach combining vision and point cloud technologies. A wide-angle camera is mounted on a crane boom and functions as an overhead camera to track moving objects at a site during lift scenarios. The 2D image frames are aligned with a laser-scanned point cloud, and the corresponding 3D bounding boxes for tracked objects are dynamically updated. The tracking and updating loops are executed asynchronously to enable real-time operation of the framework. The proposed method is validated with a mobile crane in five different lift scenarios with different moving objects such as cars and workers. The visual tracking step is found to achieve an 82-94% normalized position accuracy, and the point cloud-vision frame alignment step is found to result in a reasonable root-mean-square error of 3.9 pixels. In addition, tests with simulated 3D objects show that the estimated 3D bounding box can be positioned to within 0.1-0.4 m of the ground truth.