Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper systematically evaluates retrieval-augmented generation (RAG) for medical question answering using the MedQA USMLE benchmark and a textbook corpus. They analyze the impact of different components within the retrieval pipeline, including language models, embedding models, retrieval strategies, query reformulation, and cross-encoder reranking, across forty configurations. Results demonstrate that RAG significantly improves zero-shot accuracy, with the best configuration (dense retrieval with query reformulation and reranking) achieving 60.49% accuracy, and that domain-specialized LLMs better leverage retrieved evidence.
Domain-specific LLMs can more effectively leverage retrieved knowledge in RAG systems, but surprisingly, simpler dense retrieval methods often provide the best balance of accuracy and computational cost.
Large language models (LLMs) have demonstrated strong capabilities in medical question answering; however, purely parametric models often suffer from knowledge gaps and limited factual grounding. Retrieval-augmented generation (RAG) addresses this limitation by integrating external knowledge retrieval into the reasoning process. Despite increasing interest in RAG-based medical systems, the impact of individual retrieval components on performance remains insufficiently understood. This study presents a systematic evaluation of retrieval-augmented medical question answering using the MedQA USMLE benchmark and a structured textbook-based knowledge corpus. We analyze the interaction between language models, embedding models, retrieval strategies, query reformulation, and cross-encoder reranking within a unified experimental framework comprising forty configurations. Results show that retrieval augmentation significantly improves zero-shot medical question answering performance. The best-performing configuration was dense retrieval with query reformulation and reranking achieved 60.49% accuracy. Domain-specialized language models were also found to better utilize retrieved medical evidence than general-purpose models. The analysis further reveals a clear tradeoff between retrieval effectiveness and computational cost, with simpler dense retrieval configurations providing strong performance while maintaining higher throughput. All experiments were conducted on a single consumer-grade GPU, demonstrating that systematic evaluation of retrieval-augmented medical QA systems can be performed under modest computational resources.