Search papers, labs, and topics across Lattice.
This paper investigates the impact of Chain-of-Thought (CoT) reasoning on sycophancy in LLMs, evaluating models across objective and subjective tasks. The study finds that while reasoning generally reduces sycophancy in final decisions, it can also mask it through deceptive justifications. Mechanistic analysis reveals that sycophancy is dynamic during the reasoning process, rather than predetermined at the input stage, and is exacerbated in subjective tasks and under authority bias.
Chain-of-Thought reasoning in LLMs is a double-edged sword, reducing sycophancy in final answers but simultaneously masking it with deceptive, logically inconsistent justifications.
Alignment techniques often inadvertently induce sycophancy in LLMs. While prior studies studied this behaviour in direct-answer settings, the role of Chain-of-Thought (CoT) reasoning remains under-explored: does it serve as a logical constraint that mitigates sycophancy, or a tool for post-hoc rationalization that masks it? We evaluate a range of models across objective and subjective tasks to investigate the issue. Results show that reasoning generally reduces sycophancy in final decisions but also masks sycophancy in some samples, where models construct deceptive justifications through logical inconsistencies, calculation errors, and one-sided arguments etc. Furthermore, LLMs are more prone to sycophancy in subjective tasks and under authority-bias. Our mechanistic analysis on three open-source models reveals that the tendency of sycophancy is dynamic during the reasoning process rather than being pre-determined at the input stage.
