Search papers, labs, and topics across Lattice.
This paper explores whether low-parameter LLMs can achieve comparable word sense disambiguation (WSD) performance to high-parameter models through reasoning-driven fine-tuning. They fine-tuned eight small LLMs (<4B parameters) using the FEWS dataset, augmented with rationale-rich annotations, and a Chain-of-Thought (CoT) approach combined with neighbor-word analysis. The results show that Gemma-3-4B and Qwen-3-4B outperform medium-parameter baselines and achieve performance comparable to GPT-4-Turbo in zero-shot settings, demonstrating strong cross-domain adaptability.
Forget giant models: carefully coaxed small LLMs can now match GPT-4-Turbo on word sense disambiguation, even generalizing to completely new domains.
Word Sense Disambiguation (WSD) remains a key challenge in Natural Language Processing (NLP), especially when dealing with rare or domain-specific senses that are often misinterpreted. While modern high-parameter Large Language Models (LLMs) such as GPT-4-Turbo have shown state-of-the-art WSD performance, their computational and energy demands limit scalability. This study investigates whether low-parameter LLMs (<4B parameters) can achieve comparable results through fine-tuning strategies that emphasize reasoning-driven sense identification. Using the FEWS dataset augmented with semi-automated, rationale-rich annotations, we fine-tune eight small-scale open-source LLMs (e.g. Gemma and Qwen). Our results reveal that Chain-of-Thought (CoT)-based reasoning combined with neighbour-word analysis achieves performance comparable to GPT-4-Turbo in zero-shot settings. Importantly, Gemma-3-4B and Qwen-3-4B models consistently outperform all medium-parameter baselines and state-of-the-art models on FEWS, with robust generalization to unseen senses. Furthermore, evaluation on the unseen"Fool Me If You Can''dataset confirms strong cross-domain adaptability without task-specific fine-tuning. This work demonstrates that with carefully crafted reasoning-centric fine-tuning, low-parameter LLMs can deliver accurate WSD while substantially reducing computational and energy demands.
