Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
Across 4.7 million activation verbalizations on prompt injection and concealment benchmarks, this work investigates how to efficiently select token positions for natural language autoencoders (NLAEs) rather than decoding every position. The authors find that a ranker trained solely on chat structure outperforms internal computational metrics for position selection, eliminating the need for an initial forward pass to identify target tokens. Explaining just 5% of selected positions recovers nearly the full threat-detection yield of exhaustive decoding, while existing verbalizers zero-shot transfer to extract concepts the target model was explicitly fine-tuned to conceal.
Auditing just 5% of token positions catches nearly all model threats—and simple chat structure picks those tokens better than internal computational signals without requiring a forward pass.
Natural language autoencoders translate a language model's internal activations into readable explanations. Explaining every token position is costly. Which positions should an auditor inspect to understand a potential threat? We study this question across 4.7 million explanations on prompt injection and concealment. We compare signals from model computation with a ranker trained only on chat structure. Chat structure usually selects more relevant explanations than the computational signals, without requiring a model forward pass for position selection. On three of four datasets, explaining just 5% of positions retains nearly all of the success rate from explaining every position, where success means obtaining an explanation about the threat. The benefit varies with the audit task. We also show that pretrained verbalizers recover words that models have learned to conceal through fine-tuning, without additional verbalizer training. These results identify where auditors can concentrate explanation generation and show that useful explanations can extend beyond the model a verbalizer was trained to describe.