Search papers, labs, and topics across Lattice.
This paper introduces ThinkOmni, an omni-modal large language model framework designed for audio forgery detection and localization that addresses the limitations of existing methods which often overfit to low-level artifacts. By constructing the Forensic-Aware Chain-of-Thought (FACoT) dataset and employing Forensic-Aware Modality-Incremental Learning (FMIL), ThinkOmni explicitly integrates forensic reasoning with detection and localization tasks. The results demonstrate significant improvements in cross-dataset generalization for both spoofing detection and temporal manipulation localization, showcasing the effectiveness of the proposed methods.
ThinkOmni achieves unprecedented cross-dataset generalization in audio forgery detection by explicitly linking forensic reasoning to manipulation cues.
Existing audio forgery detection and localization (AFDL) methods often overfit dataset-specific low-level artifacts, limiting their generalization to subtle, localized, and unseen manipulations. Recent audio large language model (ALLM)-based approaches cast AFDL as question answering but still model forensic evidence implicitly, without linking manipulation cues to predictions. To bridge this gap, we propose ThinkOmni, a reasoning-driven omni-modal large language model that jointly performs explicit forensic reasoning, spoofing detection, and temporal manipulation localization. To enable explicit reasoning supervision, we construct Forensic-Aware Chain-of-Thought (FACoT), a 100K-sample dataset with structured forensic evidence and reasoning annotations. Leveraging FACoT, we introduce Forensic-Aware Modality-Incremental Learning (FMIL), which progressively aligns semantic, acoustic, and spectral-visual representations with the LLM backbone to capture complementary forensic cues. We further propose Forensic-Consistent Multi-task Loss (FCML), which combines weighted cross-entropy with an adaptive localization loss to coordinate reasoning generation, spoofing detection, and temporal localization. Extensive experiments show that ThinkOmni achieves strong cross-dataset generalization in both detection and localization. Code, models, data, and inference examples are available at https://beyond0814.github.io/ThinkOmni/.
