Search papers, labs, and topics across Lattice.
The paper introduces CMedTEB, a high-quality Chinese medical text embedding benchmark created using a multi-LLM voting pipeline validated by clinical experts, covering retrieval, reranking, and STS tasks. To address the latency issues of LLM-based retrieval, they propose CARE, an asymmetric retriever using a lightweight BERT encoder for queries and a powerful LLM encoder for documents. A novel two-stage training strategy is introduced to effectively train the asymmetric architecture, enabling CARE to outperform symmetric models on CMedTEB without increasing inference latency.
Asymmetric encoders, trained with a novel two-stage approach, can beat symmetric LLM-based models in Chinese medical text retrieval while maintaining low latency.
Effective medical text retrieval requires both high accuracy and low latency. While LLM-based embedding models possess powerful retrieval capabilities, their prohibitive latency and high computational cost limit their application in real-time scenarios. Furthermore, the lack of comprehensive and high-fidelity benchmarks hinders progress in Chinese medical text retrieval. In this work, we introduce the Chinese Medical Text Embedding Benchmark (CMedTEB), a benchmark spanning three kinds of practical embedding tasks: retrieval, reranking, and semantic textual similarity (STS). Distinct from purely automated datasets, CMedTEB is curated via a rigorous multi-LLM voting pipeline validated by clinical experts, ensuring gold-standard label quality while effectively mitigating annotation noise. On this foundation, we propose the Chinese Medical Asymmetric REtriever (CARE), an asymmetric architecture that pairs a lightweight BERT-style encoder for online query encoding with a powerful LLM-based encoder for offline document encoding. However, optimizing such an asymmetric retriever with two structurally different encoders presents distinctive challenges. To address this, we introduce a novel two-stage training strategy that progressively bridges the query and document representations. Extensive experiments demonstrate that CARE surpasses state-of-the-art symmetric models on CMedTEB, achieving superior retrieval performance without increasing inference latency.
