Search papers, labs, and topics across Lattice.
This paper explores the application of transcoders for analyzing deceptive behavior in language models, specifically using a Qwen3-4B model with per-layer transcoders (PLTs). By constructing attribution graphs to analyze feature activations and dependencies, the authors identified a set of deception-related features that significantly influence the model's outputs, leading to predictable shifts between deceptive and non-deceptive responses. The findings indicate that deception arises from internal mechanisms within the model, underscoring the potential of transcoders for enhancing behavioral monitoring and detecting security vulnerabilities in AI systems.
Deception in language models can be traced to specific internal features, revealing a pathway for proactive security measures against malicious behaviors.
Transcoders have recently emerged as a promising approach for mechanistic interpretability (MI), enabling circuit-level analysis of model behaviour. In this paper, we investigate the use of transcoders to analyse deceptive behaviour in language models, a behaviour that poses a safety and security risk. Using a Qwen3-4B model with pre-trained transcoders, specifically per-layer transcoders (PLTs), we construct attribution graphs that capture feature activations and inter-feature dependencies, allowing circuit-level analysis of deception. Through feature steering and circuit analysis, we identified a dictionary of deception-related features and show that these features exert a stronger influence on deceptive outputs, as they produce predictable shifts between deceptive and non-deceptive responses. These findings suggest that deception emerges from internal model mechanisms and highlight the potential of transcoders for behavioural monitoring and early detection of security vulnerabilities related to malicious behaviours in language models.
