Search papers, labs, and topics across Lattice.
This paper investigates the performance of small LLMs (around 1B parameters) from the Llama-3, Gemma-3, and Qwen3 families on 20 Italian clinical NLP tasks, comparing few-shot prompting, constraint decoding, fine-tuning, and continual pre-training adaptation strategies. The study finds that fine-tuning is the most effective adaptation strategy, while few-shot prompting with constraint decoding offers a strong lower-resource alternative. The best configuration, based on Qwen3-1.7B, outperforms Qwen3-32B by an average of 9.2 points, demonstrating the potential of small LLMs in medical NLP.
Small language models can not only match but significantly outperform their larger counterparts in specialized medical NLP tasks, challenging the assumption that bigger is always better.
Large Language Models (LLMs) consistently excel in diverse medical Natural Language Processing (NLP) tasks, yet their substantial computational requirements often limit deployment in real-world healthcare settings. In this work, we investigate whether "small" LLMs (around one billion parameters) can effectively perform medical tasks while maintaining competitive accuracy. We evaluate models from three major families-Llama-3, Gemma-3, and Qwen3-across 20 clinical NLP tasks among Named Entity Recognition, Relation Extraction, Case Report Form Filling, Question Answering, and Argument Mining. We systematically compare a range of adaptation strategies, both at inference time (few-shot prompting, constraint decoding) and at training time (supervised fine-tuning, continual pretraining). Fine-tuning emerges as the most effective approach, while the combination of few-shot prompting and constraint decoding offers strong lower-resource alternatives. Our results show that small LLMs can match or even surpass larger baselines, with our best configuration based on Qwen3-1.7B achieving an average score +9.2 points higher than Qwen3-32B. We release a comprehensive collection of all the publicly available Italian medical datasets for NLP tasks, together with our top-performing models. Furthermore, we release an Italian dataset of 126M words from the Emergency Department of an Italian Hospital, and 175M words from various sources that we used for continual pre-training.
