Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This study identifies a "countdown subcircuit" within the Llama-3.1-70B-Instruct model that enables it to perform various token-counting tasks, such as writing fixed-length sentences and formatting ASCII tables. By isolating this mechanism in a controlled environment and analyzing its geometric representations, the authors reveal that the subcircuit employs a consistent motif shared across different models, indicating a common underlying architecture. The research further demonstrates that this subcircuit is utilized in a range of tasks, including those where the target length is inferred from context, highlighting the potential for reverse-engineering subcircuits to enhance our understanding of model behavior generalization.
A shared "countdown subcircuit" in LLMs reveals how they generalize token-counting tasks across diverse contexts.
Writing a sentence of exactly twelve words; ending a DNA sequence at the right codon; formatting an ASCII table. These are all tasks that language models can do that requires tracking how many tokens remain before a target. In this work, we identify in Llama-3.1-70B-Instruct a general mechanism for performing these tasks: a"countdown subcircuit"that compares the current position to a goal length and estimates the time remaining until then. We first isolate a countdown subcircuit in a controlled setting, in which the model is tasked with writing a fixed-length sentence ending in a specified word. We then investigate the geometry of the representations used by the subcircuit, and find that the subcircuit uses an identical motif previously identified in a frontier LLM on a separate task, thus suggesting that this motif is shared across models. Finally, we use unsupervised probing on a natural language dataset to find a variety of other tasks where this subcircuit is used, including tasks where the goal length is inferred from context rather than explicitly stated. Our work suggests that reverse-engineering subcircuits allows us to understand how behaviors generalize from a single example to many different tasks and even models.