Search papers, labs, and topics across Lattice.
This paper introduces FeedbackTrack, a novel framework that incorporates visual-cortex-inspired cross-frame feedback into Transformer-based object tracking systems. By utilizing two lightweight feedback pathways鈥擰uery Feedback and Gate Feedback鈥擣eedbackTrack enhances the modulation of token-level queries and context-dependent features, leading to significant performance improvements. The method consistently outperforms existing temporal mechanisms across multiple benchmark configurations, achieving state-of-the-art results while maintaining a minimal increase in model parameters.
Cross-frame feedback boosts Transformer tracking performance by leveraging historical information, outperforming traditional same-frame methods by up to 3.2 AO points.
Visual object tracking requires effective temporal integration, yet most Transformer trackers still rely on predominantly feed-forward feature extraction. Existing temporal mechanisms typically update templates, prompts, queries, or prediction states, while intermediate representations are rarely reused to modulate corresponding processing stages. We propose \textbf{FeedbackTrack}, a visual-cortex-inspired framework that introduces sparse, group-level layer-aligned cross-frame feedback into pretrained Transformer trackers. Previous-frame intermediate states are detached, cached, and returned to corresponding Transformer groups in the current frame through two lightweight pathways: Query Feedback for token-level query modulation and Gate Feedback for context-dependent feature modulation. FeedbackTrack preserves the original tracking pipeline with only a fixed-size one-frame cache. Across SPMTrack and ARTrackV2, FeedbackTrack consistently improves five backbone configurations on LaSOT and GOT-10k, achieving 83.4 AO and 79.1 AUC with SPMTrack-G while adding less than 1\% parameters. Controlled comparisons show that cross-frame feedback outperforms same-frame modulation by 1.8--3.2 AO points, demonstrating that the gains mainly come from recurrent historical information. Further analysis reveals a non-uniform depth-dependent organization of learned feedback strengths, highlighting the effectiveness of recurrent feedback for Transformer tracking.
Tsinghua AI