Video-Based Activity Forecasting for Construction Safety Monitoring Use Cases

Abstract

Forecasting activities of construction workers and equipment is immensely desirable for construction safety management, as it can enable capturing near-miss of accidents and right-time intervention of collision, struck-by, trespassing, and improper use of tools. In this paper, an activity forecasting framework for construction safety management is presented, and an application to forecast workers and equipment’s motion trajectory from previously observed motion is implemented. A long short-term memory (LSTM) encoder-decoder network is developed to forecast future locations with mixture density network (MDN) used to model uncertainty in predictions. Two contextual cues are proved to help activity forecasting: (1) worker/equipment placement and distances; (2) object type attributes. A joint training schema is employed to forecast target’s locations at different future times. The proposed framework can handle very long sequences (around 2000 steps) well and accurately predict future locations in maximum 40 frames (2 seconds). Our experiment results show that the proposed model significantly outperforms conventional time-series analysis models. In 1080p high-definition videos the final model achieves average localization error in 10, 20, 40 future frames with 7.30, 12.71, and 24.22 pixels, respectively.