Search papers, labs, and topics across Lattice.
This paper introduces SURGELLM, a unified transformer framework that tackles the challenges of mismatched inductive biases, class-imbalance in feature statistics, and the lack of attention conditioning on external lexical knowledge in multi-task NLP. The framework employs a surgical feature gate, task-conditioned prefix tokens, and Instance-Weighted Normalization (IWN) to enhance model performance across four diverse tasks, achieving a macro-F1 score of 0.940, significantly outperforming non-IWN baselines. The results demonstrate that the improvements are driven by lexical alignment rather than mere parameter adjustments, underscoring the importance of feature gating in multi-task learning.
Achieving a macro-F1 score of 0.940 reveals that task-aware feature gating can dramatically enhance multi-task NLP performance by addressing class imbalance and inductive bias.
Fine-tuned encoders deployed across heterogeneous NLP tasks face three compounding problems: mismatched inductive biases, class-imbalance corruption of feature statistics, and no mechanism to condition attention on external lexical knowledge. We introduce \textbf{\surgellm}, a unified transformer framework that addresses each with a dedicated lightweight module: a \emph{surgical feature gate} (learned per-dimension sigmoid over curated lexical indicators and \texttt{[CLS]}; provably degenerates to identity when features are uninformative), \emph{task-conditioned prefix tokens} (quantized feature values and task identity prepended to every input), and \emph{Instance-Weighted Normalization} (IWN; removes class-prior bias from gate statistics). We prove an excess-risk bound linking gate benefit to \emph{surgical feature alignment}. Across four tasks, SST-2, multi-hop retrieval, LLM-prompt attribution, and authorship detection, covering 17,830 examples and eleven model variants over three seeds, the IWN variant achieves macro-F1 \textbf{0.940} ($+0.036$ over the strongest non-IWN baseline; $+0.130$ on authorship detection). A random-vocabulary control ($-0.028$ avg.\ F1) confirms gains are lexical, not parametric. Code, vocabularies, and a $99.5\%$-recovery auto-extraction recipe are released.
