Search papers, labs, and topics across Lattice.
This study introduces Guideline-as-Oracle (GAO), a novel framework that leverages a 70-row operational rule table derived from the American Academy of Ophthalmology guidelines to train a multi-turn medical triage agent without the need for costly expert annotations. By employing various construction strategies for dialogue generation, the authors successfully fine-tune a 9B parameter model, GAO-Triage, achieving significant improvements in agreement with operational references and emergent-case recall. Notably, GAO-Triage outperforms seven tested general-purpose systems, demonstrating the effectiveness of guideline-driven training in medical dialogue systems without requiring complex models at inference time.
Guideline-driven training can elevate a medical triage agent's performance without expert annotations, achieving a remarkable 74.1% agreement with operational benchmarks.
Scaling supervision for multi-turn medical agents is difficult because expert dialogue annotation is costly and clinical conversations are privacy-restricted. We introduce Guideline-as-Oracle (GAO), which compiles American Academy of Ophthalmology guidance into a 70-row operational rule table and uses it as the sole source of instance-level supervision for 3,000 training dialogues, reserving human labeling for evaluation. Because converting rules into dialogues is itself a design problem, we catalog eight construction strategies, including cited-row tier assignment, one-fact boundary pairs, metadata-only repair, and label repair, and characterize the evidential status of each: labeling mechanism, null, confounded, or evaluated only as a package. Fine-tuning a 9B backbone on this corpus yields GAO-Triage, improving agreement with a 201-case operational reference from 61.7% to 74.1% (exact McNemar p=0.0046) and emergent-case recall from 9.5% to 69.0%; the gains persist across a second seed and patient simulator. None of the seven general-purpose systems we test dominates GAO-Triage on both metrics, and GAO-Triage requires no frontier model at inference time. Permuting label-dialogue assignments collapses the model to a constant-routine predictor, indicating that the signal lies in guideline-derived assignment rather than dialogue surface form. Label repair coincides with the disappearance of a late-training safety degradation.
