Search papers, labs, and topics across Lattice.
This paper investigates the use of Amplitude Gating (AG) as an inference-time intervention for feed-forward networks (FFNs) in large language models (LLMs) to enhance structured outputs without retraining. The study reveals that while traditional Orthogonal Residual Projection (ORP) methods can inadvertently degrade performance, AG effectively modulates activation magnitudes, leading to improved performance on tool-structured tasks across multiple model variants. Notably, the introduction of learned gates results in significant performance gains, with Qwen3.5-9B achieving a 4.27 percentage point increase in agentic performance on specific tasks.
Amplitude Gating boosts tool-structured task performance in LLMs by over 7 percentage points without the need for retraining, highlighting a new avenue for inference optimization.
Large language models increasingly operate as tool-using agents, where small format, argument, or function-call errors can invalidate otherwise plausible responses. We study inference-time feed-forward network (FFN) intervention for improving structured outputs without retraining model weights. Our project began with Orthogonal Residual Projection (ORP), a direction-changing repair attempt that revealed sensitive SwiGLU FFN intervention sites but often caused more harm than fixes. We therefore propose Amplitude Gating (AG), a non-destructive alternative that preserves pretrained FFN weight directions and modulates only activation magnitudes during generation. We define a fine-grained intervention system spanning P1/P2/P3 and branch-specific P1s/P2a/P2b sites, and introduce an evaluation protocol that separates combination-oracle headroom from fixed configurations and learned gates, enforces sample-level accounting, and uses task-aware metrics for binary and partial-credit datasets. Across Qwen3.5-9B, Qwen3-8B, and Qwen2.5-7B, AG is weakly positive in aggregate but strongest on tool-structured tasks. On Qwen3.5-9B, a category-level learned gate improves tool/structured/agentic performance from 38.66% to 42.92% (+4.27 percentage points), with Hermes function-call tasks reaching about +7.6 points. On Qwen3-8B, Hermes JSON mode improves by +11.36 points. Qwen2.5-7B retains oracle headroom but current learned gates fail to capture it, showing that deployment requires model- and category-specific routing. Comparisons of entropy AG with Newton-Schulz-windowed AG show that neither family is uniformly dominant. These results identify tool-structured inference as the most credible first target for safe FFN-level inference optimization, while prospective online validation and broader cross-model evaluation remain necessary.
