A Local–Global Dual-Stream Network for Building Extraction From Very-High-Resolution Remote Sensing Images

Abstract

Buildings constitute one of the most important landscapes in remote sensing (RS) images and have been broadly analyzed in a wide range of applications from urban planning to other socioeconomic studies. As very-high-resolution (VHR) RS imagery becomes more accessible, the current building extraction methods are confronted with the challenges of the diverse appearances, various scales, and complicated structures of buildings in complex scenes. With the development of context-aware deep learning methods, it has been proven by numerous works that capturing contextual information can offer spatial relation cues for robust recognition and detection of the objects. In this article, we propose a novel local-global dual-stream network (DS-Net) that adaptively captures local and long-range information for the accurate mapping of building rooftops in VHR RS images. The local branch and the global branch of DS-Net work in a complementary manner to each other with different fields of view on the input image. Through a well-defined dual-stream architecture, DS-Net learns hierarchical representations for both the local and global branches, and a deep feature sharing strategy is further developed to enforce more collaborative integration of the two branches. Extensive experiments were carried out to verify the effectiveness of our model on three widely used VHR RS data sets: the Massachusetts buildings data set, the Inria Aerial Image Labeling data set, and the DeepGlobe Building Detection Challenge data set. Empirically, the proposed DS-Net achieves competitive or superior performance compared with the current state-of-the-art methods in terms of quantitative measures and visual evaluations.