Search papers, labs, and topics across Lattice.
The paper introduces PPT-Bench, a diagnostic benchmark to evaluate "epistemic attack" on LLMs, where prompts challenge the legitimacy of knowledge, values, or identity. They categorize philosophical pressure into four types: Epistemic Destabilization, Value Nullification, Authority Inversion, and Identity Dissolution, and test models across baseline, single-turn pressure, and multi-turn Socratic escalation conditions. Results across five models show statistically separable inconsistency patterns for each pressure type, indicating that epistemic attacks reveal vulnerabilities beyond standard social-pressure benchmarks, and that mitigation strategies are highly type- and model-dependent.
LLMs buckle under philosophical pressure, revealing distinct failure modes when their knowledge, values, or identity are challenged, not just when they face disagreement.
Large language models (LLMs) can shift their answers under pressure in ways that reflect accommodation rather than reasoning. Prior work on sycophancy has focused mainly on disagreement, flattery, and preference alignment, leaving a broader set of epistemic failures less explored. We introduce \textbf{PPT-Bench}, a diagnostic benchmark for evaluating \textit{epistemic attack}, where prompts challenge the legitimacy of knowledge, values, or identity rather than simply opposing a previous answer. PPT-Bench is organized around the Philosophical Pressure Taxonomy (PPT), which defines four types of philosophical pressure: Epistemic Destabilization, Value Nullification, Authority Inversion, and Identity Dissolution. Each item is tested at three layers: a baseline prompt (L0), a single-turn pressure condition (L1), and a multi-turn Socratic escalation (L2). This allows us to measure epistemic inconsistency between L0 and L1, and conversational capitulation in L2. Across five models, these pressure types produce statistically separable inconsistency patterns, suggesting that epistemic attack exposes weaknesses not captured by standard social-pressure benchmarks. Mitigation results are strongly type- and model-dependent: prompt-level anchoring and persona-stability prompts perform best in API settings, while Leading Query Contrastive Decoding is the most reliable intervention for open models.
