Search papers, labs, and topics across Lattice.
This paper introduces ConRub-Med, a novel reinforcement learning framework that utilizes model-generated consensus rubrics to enhance open-ended medical question answering. By employing three heterogeneous language models to independently propose criteria and a separate model to validate these criteria, the approach ensures clinically relevant feedback while minimizing expert involvement. The method outperforms existing models on multiple benchmarks, achieving the highest scores in both medical relevance and generalization, demonstrating its effectiveness in addressing the challenges of evaluating complex medical responses.
ConRub-Med achieves unprecedented accuracy in open-ended medical question answering by leveraging scalable, model-generated rubrics that outperform traditional expert-driven methods.
Reinforcement learning with verifiable rewards has been especially effective in mathematics and coding, where answers can be checked automatically. Many open-ended medical questions lack comparably cheap outcome verifiers: responses may be partly correct, incomplete, or contain clinically consequential errors. Rubrics written or validated by physicians offer strong clinical grounding, but involving experts in every instance is costly. Model-generated rubrics make this supervision scalable. We introduce ConRub-Med to preserve useful distinctions as rubric feedback moves from construction to policy optimization. For each prompt, three heterogeneous language models propose atomic criteria independently; a separate model reviews them, retaining only criteria with semantic support from all three generators. Three-State scoring distinguishes correct coverage, missing information, and incorrect claims. Errors receive negative rather than zero credit. When every response in a complete Group Relative Policy Optimization (GRPO) group receives the same final reward, a pairwise judge provides sequence advantages only if both candidate orders agree, without changing the scalar rewards. Groups without ties use vanilla GRPO. In a blinded study matched by question, two medical experts rate panels from the full pipeline as more clinically relevant than panels produced by one generator. Across the evaluated open models, ConRub-Med ranks first on six of nine benchmarks and achieves the highest medical and generalization averages. Using the resulting rubric dataset of 5,166 prompts, it scores $38.98 \pm 1.04$ (mean $\pm$ SD) on HealthBench-Hard, compared with InfiMed-ORBIT's 33.60 with 8,000 samples and 37.30 with 28,000.
Tsinghua AI