A Perspective-Embedded Scale-Selection Network for Crowd Counting in Public Transportation

Abstract

Crowd counting in congested urban transport systems is a highly challenging task for computer vision and deep learning due to several factors such as mutual occlusion, perspective change, and large-scale variations. In this paper, a novel perspective-embedded scale-selection multi-column network named PESSNet is proposed for crowd counting and high-quality density maps generation. The proposed method aligns the branches to various scales by leveraging different receptive fields, and utilizes perspective parameters to adjust the sensitivity of each branch to different perspective areas in the scene. Specifically, the PESSNet consists of four key components: 1) feature pyramid network (FPN) fuses multi-stage features extracted from the backbone network; 2) scale-selection dilated layer (SSDL) extracts features by using different dilated convolution kernels for each stage; 3) perspective-embedded fusion layer (PEFL) encodes the spatial perspective relationships across all feature levels into the network and provides a more effective fine-grained weight map; and 4) density maps generator (DMG) employs deconvolution layer as a decoder to generate high-quality density maps. The above strategies maximizes the ability of multi-column network to extract the features of instances with various scales. Extensive experiments on seven crowd counting benchmark datasets, JHU-CROWD, ShanghaiTech, UCF-QNRF, ShanghaiTechRGBD, WorldEXPO’10, TRANCOS, and NWPU-Crowd indicate that PESSNet achieves reliable recognition performance and high robustness in difference crowd counting.