Search papers, labs, and topics across Lattice.
TubeLite introduces a lightweight framework for spatio-temporal action detection that prioritizes stable tube construction and boundary-aware temporal modeling, addressing the common issues of jitter and fragmentation in frame-level detectors. By representing each actor as a tube and enforcing temporal consistency at both spatial and semantic levels, TubeLite achieves significant improvements in video-level localization performance. The method outperforms state-of-the-art approaches on the MultiSports and UCF101-24 datasets, enhancing Video-mAP@0.5 by 4.5 and 7.1 percentage points, respectively, while maintaining a compact design with fewer parameters and operations.
Achieving superior spatio-temporal action detection with a lightweight framework that outperforms heavy transformer models while reducing computational costs.
Spatio-temporal action detection in videos requires jointly localizing actors in space and identifying action boundaries over time. A common challenge is constructing temporally stable action tubes, as frame-level detectors often suffer from jitter, fragmentation, and imprecise temporal localization. Many recent approaches address this by introducing heavy spatio-temporal transformers or optical-flow-based pipelines, leading to high computational cost and limited scalability. We propose TubeLite, a lightweight framework for spatio-temporal action detection that focuses on stable tube construction and boundary-aware temporal modeling. TubeLite represents each actor as a tube, defined as a sequence of bounding boxes associated with a single actor over time, and explicitly enforces temporal consistency at both the spatial and semantic levels. The method combines low-jitter actor detection, Gaussian-weighted actor feature extraction, efficient short-term temporal propagation, and a boundary-focused temporal prediction head, while avoiding optical flow and large-scale temporal attention. Despite its compact design, TubeLite achieves strong video-level localization performance. It improves Video-mAP@0.5 by 4.5 and 7.1 percentage points over the best compared method on the MultiSports and UCF101-24 datasets, respectively, with substantially fewer parameters and floating-point operations than transformer-based alternatives, demonstrating that effective spatio-temporal action detection can be obtained through principled, lightweight temporal modeling.
