Hierarchical Convolutional Features for Visual Tracking

TLDR

Visual object tracking is difficult because targets undergo deformation, abrupt motion, background clutter, and occlusion, and while deep convolutional layers provide robust semantic representations, their coarse spatial resolution limits precise localization, whereas earlier layers offer finer localization but less invariance. This work exploits features from deep convolutional neural networks trained on object recognition to enhance tracking accuracy and robustness by interpreting the convolutional hierarchy as a nonlinear image pyramid. The authors adaptively learn correlation filters on each convolutional layer and hierarchically infer the maximum response across layers to locate the target. Extensive experiments on a large‑scale benchmark demonstrate that the proposed algorithm outperforms state‑of‑the‑art methods.

Abstract

Visual object tracking is challenging as target objects often undergo significant appearance changes caused by deformation, abrupt motion, background clutter and occlusion. In this paper, we exploit features extracted from deep convolutional neural networks trained on object recognition datasets to improve tracking accuracy and robustness. The outputs of the last convolutional layers encode the semantic information of targets and such representations are robust to significant appearance variations. However, their spatial resolution is too coarse to precisely localize targets. In contrast, earlier convolutional layers provide more precise localization but are less invariant to appearance changes. We interpret the hierarchies of convolutional layers as a nonlinear counterpart of an image pyramid representation and exploit these multiple levels of abstraction for visual tracking. Specifically, we adaptively learn correlation filters on each convolutional layer to encode the target appearance. We hierarchically infer the maximum response of each layer to locate targets. Extensive experimental results on a largescale benchmark dataset show that the proposed algorithm performs favorably against state-of-the-art methods.