Search papers, labs, and topics across Lattice.
This study investigates the hidden-state inversion problem by recovering input sequences from the last-layer hidden states of decoder-only language models, specifically focusing on a continuous embedding-space optimization approach. By avoiding hard-token projections during the search and committing tokens only at the end of the optimization loop, the authors reveal rich internal signals and a discrete loss metric that allows for effective assessment of recovery accuracy. The findings indicate a significant improvement in exact-match recovery rates from 66.9% to 97.5% for 10-token prompts, highlighting a sharp asymmetry in recovery success based on token types, with function words being more challenging to reconstruct than content-bearing tokens.
Recovering input text from hidden states reveals that function words are the primary culprits behind reconstruction failures, while content tokens are nearly perfectly retrieved.
This work studies the hidden-state inversion problem: recovering the original input token sequence of a decoder-only language model from its last-layer hidden states. Rather than treating inversion as a one-shot reconstruction, we study it as a continuous embedding-space optimisation in which a soft proxy is driven towards the leaked target without any hard-token projection during the search, and a token is committed only once, at the end of the inner loop. This design choice has two consequences which are the main focus of this paper. First, keeping the optimisation entirely in continuous space exposes a rich set of internal signals: rank trajectories of the ground-truth token, per-position loss curves, and a discrete loss measured at commit time. Second, the discrete loss allows assessing the correctness of recovery via cumulative discrete loss. We further analyse which tokens break the reconstructions and find a sharp categorical asymmetry: space-prefixed, high-frequency function words in dense regions of the embedding matrix dominate the failures, while content-bearing tokens are recovered almost perfectly. On 10-token C4 prompts the exact-match rate rises from 66.9% to 97.5% (mean similarity 0.994) as the candidate window is widened, confirming that most errors are recoverable near-misses rather than genuine ambiguities. A comparison with the released SIPIT reference situates these findings: per-step hard projection is faster, but the continuous formulation is what makes the optimisation observable and its failures detectable. The results show that last-layer hidden states of GPT-2 are as sensitive as the original text.
