Search papers, labs, and topics across Lattice.
The paper introduces ACROS, a method for inducing explicit sense representations into frozen pretrained decoder LMs via a gated residual addition. ACROS enables zero-shot word-sense disambiguation, low-KL lexical steering, and cross-lingual adaptation, all while preserving the base LM's performance. Experiments on SmolLM2-360M demonstrate strong performance across these tasks, suggesting ACROS effectively creates an inducible interface for sense representations in standard LMs.
Now you can retrofit any frozen language model with explicit, editable "sense knobs" for steering, disambiguation, and cross-lingual transfer, without pretraining from scratch.
Sense representations (explicit, per-token meaning decompositions) are useful for disambiguation, steering, and cross-lingual alignment, but existing approaches require models to be pretrained with sense structure baked in. We introduce ACROS, which induces an explicit sense pathway into a frozen pretrained decoder LM through a gated residual addition. On SmolLM2-360M, ACROS preserves base LM quality while supporting three uses of the same induced variables: zero-shot word-sense disambiguation (64.95 F1 on Raganato ALL, competitive with the WordNet first-sense heuristic), low-KL lexical steering across 5,161 CoInCo cases where a simple non-oracle proxy recovers about 90% of positive shifts, and SENSIA cross-lingual adaptation to four languages (mean R@1 0.988, target FLORES PPL 7.94). ACROS makes sense representations an inducible interface for ordinary pretrained LMs.
