Search papers, labs, and topics across Lattice.
This paper tackles word sense plausibility rating by comparing embedding-based methods, transformer fine-tuning, and LLM prompting. The best results are achieved with a structured prompting strategy that decomposes the narrative and uses explicit decision rules for rating calibration. The analysis demonstrates that structured prompting outperforms fine-tuning and embedding methods, with prompt design being more critical than model scale.
Forget bigger models: clever prompt engineering with explicit decision rules crushes fine-tuning and embeddings for word sense disambiguation.
Word sense plausibility rating requires predicting the human-perceived plausibility of a given word sense on a 1--5 scale in the context of short narrative stories containing ambiguous homonyms. This paper systematically compares three approaches: (1) embedding-based methods pairing sentence embeddings with standard regressors, (2) transformer fine-tuning with parameter-efficient adaptation, and (3) large language model (LLM) prompting with structured reasoning and explicit decision rules. The best-performing system employs a structured prompting strategy that decomposes evaluation into narrative components (precontext, target sentence, ending) and applies explicit decision rules for rating calibration. The analysis reveals that structured prompting with decision rules substantially outperforms both fine-tuned models and embedding-based approaches, and that prompt design matters more than model scale for this task. The code is publicly available at https://github.com/tongwu17/SemEval-2026-Task5.
