Search papers, labs, and topics across Lattice.
This paper introduces ST$^2$U, a novel approach to test-time unlearning in large language models that addresses the limitations of existing activation-editing methods by implementing trajectory-wide boundary control. By modeling restricted knowledge boundaries in low-dimensional invertible coordinates and propagating historical correction states, ST$^2$U effectively prevents the re-entry of restricted knowledge during autoregressive generation. The method achieves a significant reduction in restricted-knowledge re-entry rates across multiple benchmarks while maintaining competitive performance in knowledge retention and inference efficiency.
ST$^2$U reduces restricted-knowledge re-entry by up to 45% while preserving model capabilities, reshaping how we approach test-time unlearning in language models.
Controlling restricted knowledge in large language models is essential for model alignment and safe deployment. Test-time unlearning avoids costly retraining and parameter updates by intervening only during inference. However, existing activation-editing methods apply isolated pointwise corrections, overlooking how autoregressive generation continually reconstructs hidden states from the prompt, cache, and generated prefix. Consequently, later states may return to restricted knowledge regions after a locally successful correction, causing restricted knowledge re-entry. In this work, we propose Stateful Test-Time Unlearning via restricted knowledge boundary control (ST$^2$U), which formulates test-time unlearning as trajectory-wide boundary control. ST$^2$U first models restricted knowledge boundaries in low-dimensional invertible coordinates while leaving orthogonal non-target components unchanged. During inference, ST$^2$U monitors risk along the trajectory, applies minimal boundary corrections with contextual anchoring, and propagates historical correction states across tokens to mitigate knowledge re-entry. This trajectory-wide control enables more persistent forgetting while preserving non-target capabilities and limiting inference overhead. Across three benchmarks and three model families, ST$^2$U delivers the strongest overall balance, combining best or second-best retention with competitive forgetting and substantially less restricted-knowledge re-entry than test-time baselines (13.76%-19.84% versus 46.50%-59.10%).
