Search papers, labs, and topics across Lattice.
This paper introduces a framework for privacy-sensitive clinical information extraction from dental notes using small language models (SLMs). The framework enables SLMs to self-generate, verify, refine, and evaluate entity-specific prompts, followed by QLoRA-based supervised fine-tuning and direct preference optimization (DPO). Experiments on 1,200 annotated notes showed that Qwen2.5-14B-Instruct and Llama-3.1-8B-Instruct achieved strong performance after DPO, with micro/macro F1 scores of 0.864/0.837 and 0.806/0.797, respectively.
Forget massive models: small, locally-deployable language models can achieve surprisingly strong performance on privacy-sensitive clinical information extraction tasks with self-prompting and preference-based optimization.
Clinical named entity recognition from dental progress notes is challenging because documentation is highly unstructured, domain-specific, and often privacy-sensitive. We developed a locally deployable framework that enables small language models to self-generate, verify, refine, and evaluate entity-specific prompts for extracting multiple clinical entities from dental notes. Using 1,200 annotated notes, we evaluated candidate open-weight models with multi-prompt ensemble inference and further adapted selected models using QLoRA-based supervised fine-tuning and direct preference optimization. Model performance varied substantially, highlighting the need for task-specific evaluation rather than reliance on generic benchmarks. Qwen2.5-14B-Instruct achieved the strongest baseline performance. After DPO, Qwen2.5-14B-Instruct and Llama-3.1-8B-Instruct achieved micro/macro F1 scores of 0.864/0.837 and 0.806/0.797, respectively. These findings suggest that automated prompt optimization combined with lightweight preference-based post-training can support scalable clinical information extraction using locally deployed small language models.
