Search papers, labs, and topics across Lattice.
This paper introduces a multitask prompt distillation and decomposition framework to learn a single shared "metaprompt" from 21 clinical NLP tasks. The metaprompt is then adapted to new target tasks using minimal trainable parameters, achieving parameter-efficient transfer learning. Experiments across diverse clinical tasks and models (including LLaMA, Meditron, and gpt-oss) demonstrate that this approach outperforms LoRA and single-task prompt tuning while using orders of magnitude fewer parameters.
Clinical NLP models can now achieve better transfer learning performance than LoRA with orders of magnitude fewer parameters by distilling knowledge into a single, shared "metaprompt."
Existing prompt-based fine-tuning methods typically learn task-specific prompts independently, imposing significant computing and storage overhead at scale when deploying multiple clinical natural language processing (NLP) systems. We present a multitask prompt distillation and decomposition framework that learns a single shared metaprompt from 21 diverse clinical source tasks and adapts it to unseen target tasks with fewer than 0.05% trainable parameters. Evaluated across five clinical NLP task types (named entity recognition, relation extraction, question answering, natural language inference, and summarization) on 10 held-out target datasets using three backbone models (LLaMA 3.1 8B, Meditron3 8B, gpt-oss 20B), our framework consistently outperforms LoRA by 1.5~1.7% despite using orders of magnitude fewer parameters, and exceeds single-task prompt tuning by 6.1~6.6%. The gpt-oss 20B model achieves the highest overall performance, particularly on clinical reasoning tasks. The strong zero- and few-shot performance demonstrates better transferability of the shared prompt representation.
