SiamTDR: Time-Efficient RGBT Tracking via Disentangled Representations

Abstract

The growing demand for vision tasks utilizing RGB-T (Red-Green-Blue-Thermal) imagery is attributed to the advantageous synergistic effect of combining RGB images with thermal (Tir) image information. Due to their exceptional real-time inference efficacy, siamese networks have garnered considerable attention in RGB-T object tracking as a leading solution. However, current RGB-T Siamese trackers still need to catch up with online training RGB-T trackers regarding accuracy and robustness due to ineffective utilization of valid information from both modes. To this end, this work proposes SiamTDR, a high-speed Siamese network-based RGB-T tracker with a disentangled representation and deconstructed features. Firstly, we introduce a single-modal feature extraction network into the Siamese network to capture cross-level information within unimodal features extracted from RGB or Tir images. Next, we employ a disentangled representation multi-modal feature fusion module (DP-MF) to extract cross-modal information between RGB and thermal features, thereby improving the information utilization of both modalities. Finally, a dual branch fusion module (DBF) significantly enhances the robustness of our tracker in the final bounding box selection stage. Besides, we also employ data augmentation techniques such as central random offset. Extensive experiments conducted on two RGB-T tracking benchmark datasets demonstrate the superior performance of our method, which achieves a tracking speed of over 127 frames per second (FPS) on the GTOT dataset.