Search papers, labs, and topics across Lattice.
RAG-Match, a three-stage framework, is introduced to improve semantic relevance judgment in knowledge-intensive search scenarios. It combines knowledge-augmented pretraining, hierarchical reasoning alignment, and preference-based decision calibration. Experiments on a real-world benchmark demonstrate that RAG-Match outperforms strong LLM baselines, showcasing the benefits of knowledge injection, reasoning supervision, and preference optimization.
Knowledge injection, reasoning supervision, and preference optimization can be combined to substantially improve semantic relevance judgment, outperforming even strong LLM baselines.
Semantic relevance judgment for search is particularly challenging in knowledge-intensive scenarios, where accurate ranking requires not only semantic matching but also background grounding, multi-step reasoning, and well-calibrated decision boundaries. Existing relevance models mainly rely on direct label supervision or shallow semantic similarity, which limits their ability to handle implicit intent, factual equivalence, and fine-grained relevance distinctions. To address this issue, we propose \textsc{RAG-Match}, a three-stage framework that integrates knowledge-augmented pretraining, hierarchical reasoning alignment, and preference-based decision calibration for relevance modeling. The key idea is to first strengthen query-centered semantic grounding, then align the model with structured relevance reasoning, and finally correct decision-level inconsistencies in difficult boundary cases. Experimental results on a real-world search relevance benchmark show that \textsc{RAG-Match} consistently outperforms strong LLM-based baselines across multiple ranking metrics, demonstrating the effectiveness of combining knowledge injection, reasoning supervision, and preference optimization for fine-grained relevance judgment.
DAMO