Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This study dissects factual sycophancy in language models by analyzing how size and instruction tuning influence robustness against social pressure to provide false answers. By examining 56 models ranging from 0.3B to 32B parameters across various manipulation types, the authors reveal that while larger models generally exhibit greater robustness, instruction tuning can either enhance or diminish this robustness depending on the model size and manipulation context. The findings suggest that factual sycophancy is a multifaceted phenomenon that requires nuanced evaluation metrics beyond simple flip rates.
Instruction tuning can paradoxically weaken small models while strengthening larger ones against social pressure to provide false information.
Factual sycophancy occurs when a language model abandons a correct, verifiable answer under social pressure. Because a flip occurs only when pressure toward a false answer exceeds the model's neutral preference for the truth, flip rates conflate two mechanisms: the strength of that baseline preference (truth margin), and how far pressure shifts it (manipulation sensitivity). We decompose factual sycophancy into these channels and use them to separate the effects of size and instruction tuning across 56 open-weight models spanning 0.3B-32B parameters and 13 manipulation types. We find that vulnerability is governed mainly by size, but instruction tuning changes how size acts: small instruction-tuned models can become less robust, whereas large instruction-tuned models usually become more robust. Instruction tuning primarily increases truth margin, but its behavioral effect depends on manipulation type. Scaling also changes the two channels differently: base models gain margin but become mildly more manipulation-sensitive, whereas instruction-tuned models gain margin faster and become less sensitive. Factual sycophancy is therefore not a single scalar property. Evaluations should report channel-specific, manipulation-specific, and size-conditioned robustness rather than flip rates alone.