Search papers, labs, and topics across Lattice.
The paper investigates the potential for large language models (LLMs) to use steganography to conceal their reasoning processes (Chain-of-Thought, CoT) from monitoring systems. It systematically evaluates the steganographic capabilities of 28 LLMs, measuring monitor evasion, refusal rates, encoding fidelity, and hidden task accuracy across four datasets using steganographic acrostics. The study finds that while current models struggle with complex tasks, Claude Opus 4.5 achieves high accuracy (92%) on a simplified counting task, demonstrating a nascent steganographic capability, and GPT-5.2 shows occasional paradoxical behavior.
LLMs are showing a surprising ability to hide their reasoning using steganography, raising concerns about the reliability of current safety monitoring techniques.
Monitoring chain-of-thought (CoT) reasoning is a foundational safety technique for large language model (LLM) agents; however, this oversight is compromised if models learn to conceal their reasoning. We explore the potential for steganographic CoT -- where models hide secret reasoning within innocuous text -- to inform risk assessment and deployment policies. We systematically evaluate the limits of steganographic capabilities across 28 models, ranging from past generations to the current frontier. We measure monitor evasion, refusal rates, encoding fidelity, and hidden task accuracy across four datasets, comparing steganographic acrostics against plain reasoning and filler-token baselines. We find that current models cannot yet sustain hidden reasoning for complex math and arithmetic tasks. However, in a simplified counting experiment, Claude Opus 4.5 achieved 92% accuracy on the hidden task, demonstrating nascent capability. Notably, in rare cases (<1%), GPT-5.2 might refuse steganographic instructions while simultaneously complying with them. Our findings underscore the need for continuous evaluation of steganographic risks. This study provides a methodology to preemptively detect and prevent hidden reasoning that might empower misaligned scheming and deceptive behavior.
