Search papers, labs, and topics across Lattice.
This paper investigates the vulnerability of activation-based detection methods for steganographic payloads in large language models (LLMs) and proposes a targeted data-level intervention to enhance detectability. By adversarially fine-tuning steganographic trojans across multiple LLMs, the authors demonstrate that while these models can evade traditional detection methods, they can still retain a significant level of secret recovery. The study reveals that a recontextualization dataset can effectively restore detection capabilities, highlighting the interplay between secret recoverability and the model's representational degrees of freedom.
Evasive steganographic payloads in LLMs can be detected again by strategically recontextualizing the data, even after successful evasion of traditional methods.
Large language models can be fine-tuned to encode prompt-borne secrets into fluent, seemingly benign outputs. This creates a steganographic exfiltration risk that is difficult to detect with output-level steganalysis. Recent work proposes mechanistic detection using linear probes that recover the secret from internal activations. We show that this defense can be systematically evaded, but that detectability can be recovered through a targeted data-level intervention. First, we extend the detection setup to include a non-linear MLP probe. We then adversarially fine-tune steganographic trojans across five base models: Qwen3-8B, Llama-3.1-8B, Ministral-8B, Qwen3-14B, and Phi-4-14B. The resulting models retain $58$--$79\%$ exact-match secret recovery while evading both ridge and held-out MLP probes, with $1$--$8\%$ average capability degradation across six benchmarks. We then give an information-theoretic characterization of this evasion. Successful evasion preserves recoverability while reducing low-order extractability of the secret from the content-aligned representation, forcing the payload into synergistic interaction with residual degrees of freedom. This motivates a recontextualization dataset that restricts these residual degrees of freedom. On this distribution, both ridge and MLP detectability are restored across all five evasive trojans. Overall, our findings show that activation-based steganography detection is vulnerable to adaptive evasion, but also that theory-guided evaluation distributions can expose otherwise hidden payloads.
ETH