Search papers, labs, and topics across Lattice.
This paper investigates the impact of prompt optimization on the performance of both small and large language models for error detection in medical text, a critical task for improving patient safety. The authors apply Genetic-Pareto (GEPA) for automatic prompt optimization and demonstrate significant improvements in error detection accuracy across models like GPT-3.5 and Qwen3-32B. The optimized prompts led to state-of-the-art performance on the MEDEC benchmark, approaching the accuracy of medical professionals.
LLMs can spot medical errors almost as well as doctors, but only with the right prompt – and Genetic-Pareto optimization can find it automatically.
Errors in medical text can cause delays or even result in incorrect treatment for patients. Recently, language models have shown promise in their ability to automatically detect errors in medical text, an ability that has the opportunity to significantly benefit healthcare systems. In this paper, we explore the importance of prompt optimisation for small and large language models when applied to the task of error detection. We perform rigorous experiments and analysis across frontier language models and open-source language models. We show that automatic prompt optimisation with Genetic-Pareto (GEPA) improves error detection over the baseline accuracy performance from 0.669 to 0.785 with GPT-5 and 0.578 to 0.690 with Qwen3-32B, approaching the performance of medical doctors and achieving state-of-the-art performance on the MEDEC benchmark dataset. Code available on GitHub: https://github.com/CraigMyles/clinical-note-error-detection
