Search papers, labs, and topics across Lattice.
This paper introduces UniSkip-Mamba, a frequency-aware State Space Model designed to enhance Audio-Visual Temporal Forgery Localization (AV-TFL) by focusing on low/mid-frequency components that are critical for detecting forgery patterns. The authors demonstrate that existing methods struggle with high-frequency noise, and by systematically analyzing frequency domains, they show that removing these components can improve detection performance by 1.4%. Their approach achieves state-of-the-art results, with significant improvements in average precision and mean average precision, while also enabling six times faster inference.
By focusing on low/mid-frequency patterns, UniSkip-Mamba outperforms existing models in forgery detection, achieving a remarkable 63.4% AP@0.95 on LAV-DF.
With the proliferation of AI-generated content, sophisticated multimedia manipulation has raised critical concerns about malicious applications such as opinion manipulation and evidence fabrication, making Audio-Visual Temporal Forgery Localization (AV-TFL) an urgent research frontier. Existing TFL methods have progressed along two main paradigms: Transformer-based temporal modeling and channel-wise multimodal fusion. While these approaches capture temporal dependencies and cross-modal correlations, they process all frequency components indiscriminately, leading to overfitting on high-frequency noise and limited robustness under real-world data degradation. Through systematic frequency domain analysis, we find that forgery-discriminative patterns concentrate in the low/mid-frequency range (normalized frequency 0-0.15), while high-frequency components primarily introduce noise, removing them even improves detection performance by +1.4%. Based on this phenomenon, we propose UniSkip-Mamba, a frequency-aware State Space Model framework that incorporates Unified Multimodal Sequence Fusion to preserve cross-modal phase relationships, and Skip-Scanning Mamba Blocks that implement frequency-aware regularization through a novel Group-Scan-Merge mechanism, naturally biasing learning toward discriminative low/mid-frequency patterns (0-0.15) while maintaining representational completeness. We achieve state-of-the-art (SOTA) performance: 63.4% AP@0.95 on LAV-DF (+9.8% improvement) and 63.58% mAP on AV-Deepfake1M (+14.32% improvement), with 6x faster inference. Our frequency-domain analysis provides theoretical justification from a signal processing perspective for why skip-scanning inherently improves both accuracy and robustness.
