Search papers, labs, and topics across Lattice.
This paper introduces Policy Iteration with Human Feedback (PIHF), a novel framework that enhances in-context learning by integrating expert feedback into the policy revision process of pretrained language models. By utilizing a language-model critic alongside clinical experts, the approach effectively identifies and addresses recurrent failures in reasoning and tool use, leading to significant improvements in diagnostic accuracy for ultra-rare diseases. The results demonstrate a remarkable increase in Recall@1, with gains of 32.7 percentage points for GPT-5.4 and 31.1 points for Qwen3.6-35B, showcasing the potential of expert-guided policy development in medical applications.
Expert-guided policy revisions in language models can drastically improve diagnostic accuracy, achieving up to a 32.7 percentage point increase in Recall@1 for rare diseases.
Generative pretraining established reusable task representations; later work on language-based task conditioning and in-context learning showed that a fixed model could adapt its behavior from instructions and demonstrations. Policy Iteration with Human Feedback (PIHF) builds on this development and the recurrent evaluate-and-improve structure of generalized policy iteration. PIHF uses a pretrained language model as its execution substrate and moves persistent revision to a versioned natural-language policy and tool set. A language-model critic and clinical expert review complete-panel reasoning and tool-use trajectories to localize recurrent failures and form candidate revisions; the expert may reinterpret the evidence and retains authority over admission and rollback, while Recall@1 and Recall@5 validate outcomes after candidate execution. Across cumulative ablations and ultra-rare-disease benchmarks, a PIHF-derived policy improved Recall@1 in one proprietary executor and three open-weight executors spanning 3 to 49 billion active parameters. Gains were 32.7 percentage points for GPT-5.4 and 31.1 points for Qwen3.6-35B, a difference of 1.7 points. These results support the feasibility of using pretrained language models as fixed-weight execution substrates for expert-guided policy development in rare-disease diagnosis.
