M3S-NIR: Multi-modal Multi-scale Noise-Insensitive Ranking for RGB-T Saliency Detection

Abstract

RGB-Thermal saliency detection is to use thermal infrared information to assist salient object detection with visible light information. Multi-Modal Multi-Scale Noise-Insensitive Ranking (M3S-NIR), is proposed for RGB-Thermal (RGB-T) saliency detection. Given spatially aligned RGB and thermal images, M3S-NIR first segments them together into a set of multi-scale superpixels. Second, it takes these superpixels as graph nodes and performs multi-modal multi-scale manifold ranking to achieve saliency calculation, in which the cross-modal and cross-scale collaborations are performed to integrate different kinds of information. Third, to handle noises and corruptions of ranking seeds (i.e., boundary superpixels) introduced by salient objects and RGB-T alignment, M3S-NIR introduces an intermediate variable to infer the optimal ranking seeds, and formulates it as a sparse learning problem. Finally, M3S-NIR uses a unified ADMM (Alternating Direction Method of Multipliers)-based optimization framework to solve the ranking model efficiently. Extensive experiments on the benchmark dataset demonstrate the effectiveness of the proposed approach over other state-of-the-art RGB-T saliency detection methods.