Search papers, labs, and topics across Lattice.
This study investigates the fragility of large language models' reasoning capabilities by employing a technique called Concept Chaining, which biases model predictions through implicit reasoning steering. By generating connection paragraphs that link question entities to target answers via intermediate concepts, the authors continue pretraining on these texts to assess shifts in answer preferences on multiple-choice questions. The findings reveal that this method can effectively redirect model decisions using natural language, highlighting a significant vulnerability in the reasoning processes of these models.
Implicit reasoning steering can covertly amplify latent biases in language models, shifting their predictions through seemingly innocuous text.
Large language models often appear to reason reliably, yet on many questions repeated sampling yields both correct and incorrect answers, revealing an underlying fragility in how final decisions are formed. We study whether this fragility can be exploited through implicit reasoning steering: using natural-language text to bias a model toward a designated answer without explicit instructions, triggers, or direct answer cues. Our approach, Concept Chaining, generates a short connection paragraph that links question entities to a target option through one or two intermediate concepts. We then continue pretraining a victim model on these connection paragraphs and evaluate whether its answer preference shifts on the original multiple-choice questions. Our results show that indirect, natural-looking text can systematically steer model predictions while remaining substantially less inferable than direct paraphrases, which shows that reasoning brittleness is not merely an evaluation artifact: it creates a practical channel through which latent biases can be amplified by ordinary-looking text to covertly redirect model decisions.
