Search papers, labs, and topics across Lattice.
This study investigates the monitorability of large language models (LLMs) using chain-of-thought (CoT) reasoning, particularly focusing on the implications of latent CoT approaches that sacrifice explicit token traces for efficiency. By employing a hint-based intervention setup, the authors assess how different reasoning modes affect the ability to monitor model behavior, revealing that task properties and access to model internals play a more critical role than the reasoning mode itself. The findings indicate that while latent CoT methods reduce inference costs, they complicate the monitoring process, necessitating alternative strategies to maintain oversight of model decision-making.
Monitorability of LLMs hinges more on task characteristics and internal access than on the reasoning mode used, challenging assumptions about CoT efficiency.
Chain-of-thought (CoT) reasoning offers a window into the decision-making of large language models (LLMs), which can be monitored for target behaviors by reading the reasoning trace, motivating work on CoT monitorability. Latent CoT approaches, however, replace the explicit tokens with a small number of continuous states, lowering inference costs but removing the readable trace this monitoring relies on. Monitoring then requires alternative access to the model, such as probing its activations or verbalizing the latent states back into text, but how much monitorability these alternatives preserve is unclear. We study this question with a hint-based intervention setup, a proxy for behaviors where models exploit biasing input cues, e.g., an inadvertently leaked answer or a belief stated by the user, without acknowledging them. Taking hint-reliance as the monitorability target, we compare monitors across reasoning modes, from explicit CoT to weakly- and strongly-supervised latent CoT, on math reasoning and question answering. We find that, in this setup, monitorability depends more on properties of the task (such as whether the correct answer constrains the supporting reasoning) and the level of access to model internals than on the reasoning mode.
