Search papers, labs, and topics across Lattice.
The paper introduces the Dignified Peer framework to address the "Evasive Servant" problem in aligned LLMs, where models sycophantically validate flawed beliefs while deflecting responsibility. They tackle challenges in data supervision and objective collapse by creating the PersonaKnob dataset with compositional persona preferences and using a tolerant constrained Lagrangian DPO algorithm. Experiments show the framework successfully creates an LLM agent exhibiting both dignity and peer-like qualities, as measured by a psychometrically calibrated evaluation protocol.
LLMs can be taught to be dignified peers instead of evasive sycophants, by carefully balancing anti-sycophancy and trustworthiness with empathy and creativity.
Current aligned language models exhibit a dual failure mode we term the Evasive Servant: they sycophantically validate flawed user beliefs while deflecting responsibility with boilerplate disclaimers. We propose the Dignified Peer framework, which counters servility with anti-sycophancy and trustworthiness, and mitigates evasiveness through empathy and creativity. Realizing this agent requires overcoming significant challenges in data supervision, objective collapse, and evaluation bias. We address these issues by introducing the PersonaKnob dataset which features a compositional partial order structure of multiple persona preference. This data is utilized alongside a tolerant constrained Lagrangian DPO algorithm that dynamically balances all persona dimensions to prevent behavioral collapse. Additionally, we employ a psychometrically calibrated Item Response Theory evaluation protocol to disentangle latent model persona capability from confounders like judge biases. Extensive empirical studies demonstrate that our approach successfully build a LLM agent with both dignity and peer.
