Search papers, labs, and topics across Lattice.
This paper introduces PRISM, a novel framework for evaluating persona fidelity in large language models (LLMs) by reformulating the task as a structured inverse inference problem. By decomposing persona fidelity into three functional dimensions鈥擳ask Framing, Interpersonal Stance, and Linguistic Style鈥擯RISM offers a more nuanced and context-sensitive assessment compared to existing methods that rely on holistic judgments or static inventories. Experimental results demonstrate that PRISM significantly improves the accuracy and stability of persona fidelity evaluations, making it a valuable tool for simulating diverse human characters in AI applications.
PRISM reveals that a structured approach to persona fidelity evaluation can drastically outperform traditional methods, providing more reliable insights into LLM behavior.
As large language models are increasingly deployed to simulate diverse human characters, ensuring persona fidelity, defined as the extent to which an agent's behavior consistently reflects the psychological and stylistic characteristics of a target persona, has become a critical requirement. However, existing evaluation paradigms primarily rely on either holistic LLM-based judges, which are prone to"holistic appraisal hallucination'', or static psychometric inventories, which fail to capture the context-dependent fidelity required in dynamic dialogue. To address these limitations, we propose PRISM (Persona Reasoning with Inverse SFL-based Modeling), a psycholinguistically grounded framework that reformulates persona fidelity evaluation as a structured inverse inference task. Inspired by Systemic Functional Linguistics (SFL), PRISM decomposes persona fidelity into three functional dimensions: Task Framing, Interpersonal Stance, and Linguistic Style. It estimates dimension-specific evidence over a persona-conditioned label space and aggregates these signals into an interpretable and auditable evaluation process. Experiments show that PRISM yields more accurate and stable judgements than traditional holistic judging, providing a more reliable framework for persona fidelity evaluation.
