Search papers, labs, and topics across Lattice.
This paper introduces FORTE, a unified framework that enhances text-to-audio retrieval by integrating structured logical reasoning with efficient cross-modal alignment techniques. By transforming queries into first-order logic and refining them through a constrained search, FORTE achieves improved semantic alignment and retrieval precision, particularly in fine-grained scenarios. Experimental results on datasets like AudioCaps and Clotho show that FORTE consistently outperforms existing models, underscoring the value of combining symbolic reasoning with representation learning in cross-modal tasks.
Fine-grained text-to-audio retrieval improves significantly when structured logical reasoning meets efficient cross-modal alignment.
Text-to-audio retrieval has made significant progress with shared embedding models such as CLAP and Pengi, yet they often struggle with fine-grained semantic alignment due to the inherent modality gap between text and audio. In this work, we propose FORTE, a unified framework that integrates structured logical reasoning with parameter-efficient cross-modal alignment to improve retrieval precision. Our approach first transforms queries into first-order logic and refines them via a constrained search that preserves semantic invariance while introducing discriminative attributes. The refined representation is then aligned with audio embeddings using a lightweight projection module, followed by a predicate-aware re-ranking step that enforces logical consistency at inference. Extensive experiments on AudioCaps and Clotho demonstrate consistent improvements over strong baselines, particularly in challenging fine-grained scenarios. Our results highlight the effectiveness of combining symbolic reasoning with representation learning for cross-modal retrieval.
