Search papers, labs, and topics across Lattice.
This paper investigates the robustness of Chain-of-Thought (CoT) prompting for code generation (LLM4Code) under various input perturbations, using token-level uncertainty to analyze reasoning trajectory stability. The study identifies that CoT's effectiveness hinges on the stability of "structural anchors" (reasoning-code transition, symbolic commitment, algorithmic articulation) within the reasoning process. The authors find that CoT's benefits are not uniform, and specific trajectory deformations (Lengthening, Branching, Simplification) emerge when perturbations interact with these anchors, leading to failure patterns diagnosable by early-stage uncertainty.
CoT's code generation prowess isn't a sure thing: its robustness crumbles when input tweaks destabilize key "structural anchors" in the reasoning process.
Chain-of-Thought (CoT) prompting is widely used to elicit explicit reasoning from large language models for code (LLM4Code). However, its impact on robustness and the stability of reasoning trajectories under realistic input perturbations remains poorly understood. Prior work has largely evaluated CoT through final correctness, leaving a critical gap in understanding how CoT reshapes internal uncertainty dynamics and why it sometimes harms rather than helps code generation. We suggest that CoT is not uniformly beneficial; instead, its robustness depends on whether perturbations destabilize structurally sensitive commitment points along the reasoning-to-code trajectory. We conduct a controlled, large-scale empirical study of CoT across six models and two code benchmarks (MHPP and BigCodeBench), subjecting task docstrings to systematic character-, word-, and sentence-level perturbations. We instrument full generation traces with token-level uncertainty and define three novel structural anchors: reasoning-code transition, symbolic commitment, and algorithmic articulation. Findings: (1) CoT does not yield uniform performance or robustness gains: its benefits are contingent on model family, task structure, and prompt explicitness. (2) CoT and No-CoT exhibit distinct robustness profiles, with different perturbation families triggering different failure modes. (3) We identify three recurrent trajectory deformations--Lengthening, Branching, and Simplification--that systematically emerge when perturbations interact with structural anchors and explain failure patterns. (4) Early-stage uncertainty serves as a reliable diagnostic signal for localizing where trajectory instability begins around sensitive anchors. These results provide a unified explanation for CoT's mixed performance and suggest design principles for building more robust reasoning-based code generators.