Search papers, labs, and topics across Lattice.
This paper introduces the Epistemic Stance Flexibility Probing (ESFP) benchmark, designed to evaluate how well language models distinguish between expert beliefs and their own opinions on contested claims. The study reveals that epistemic flexibility is largely independent of overall model capability, with a 27B open-weight model performing comparably to top proprietary systems, while some flagship models underperform. Key findings indicate that stance content density is a more reliable indicator of epistemic flexibility than surface-level lexical markers, highlighting the complexity of measuring model responses to different epistemic registers.
Epistemic flexibility in language models is not tied to their overall capability, challenging assumptions about model performance across tasks. WHY_IT MATTERS: This insight could redefine evaluation metrics for conversational agents, emphasizing the importance of nuanced epistemic responses over general competence.
A language model may be asked either what experts believe about a contested claim or what it believes about the claim itself. A trustworthy conversational agent should distinguish these two requests and respond in different epistemic registers: neutral attribution in the first case and stance expression in the second. Whether such a shift occurs-and whether it occurs coherently-is not directly assessed by existing benchmarks for accuracy, instruction following, or safety. We introduce ESFP, a behavioral benchmark that treats the contrast between externally attributed and self-attributed prompts as the fundamental unit of measurement. ESFP consists of 104 carefully controlled items spanning six epistemic categories and five phrasing templates, and evaluates model responses along four complementary dimensions: lexical self-attribution, representation-level responsiveness to role framing, sentence-level stance content density assessed by an LLM judge panel, and cross-condition stance consistency. Evaluating eight frontier models from five vendors, we find that epistemic flexibility is largely orthogonal to general model capability: a 27B open-weight model matches the strongest proprietary systems, the flagship model of one family underperforms its lightweight counterpart, and reasoning-optimized models do not consistently exhibit higher flexibility. Stance content density provides the strongest signal, while surface-level lexical markers such as'I think'can change substantially without corresponding changes in expressed stance. We provide item-level bootstrap confidence intervals, weight-sensitivity analyses, and an explicit discussion of the interpretation limits of the composite score. ESFP measures a model's propensity to adapt its epistemic stance under changing attribution conditions, rather than a general competence measure.
