Search papers, labs, and topics across Lattice.
ActTraitBench, a new human-grounded benchmark, is introduced to measure the consistency between LLMs' stated personality traits and their actual behavioral decisions. The benchmark uses one-to-one mappings between psychometric facets and behavioral paradigms, along with distributional calibration to align LLM-judge scores with human norms. Experiments on 14 LLMs reveal a significant "knowledge-decision gap," where larger models sometimes show greater behavioral divergence despite consistent self-reports; a "Chain of Cognitive Alignment" (CoCA) method is proposed to mitigate this gap.
LLMs can convincingly *say* they're conscientious, but ActTraitBench reveals they often *act* otherwise, exposing a critical gap between knowledge and behavior that scales *worse* with model size.
While Large Language Models (LLMs) can convincingly simulate personas in explicit self-reports, they often deviate in implicit behavioral decisions, revealing a substantial Knowledge-Decision Gap ($G_{\text{KD}}$). Existing benchmarks struggle to measure this asymmetry due to limited construct validity, multi-dimensional entanglement, and distributional biases in LLM-based evaluation. To address these issues, we propose ActTraitBench, a human-grounded evaluation framework for measuring personality consistency in LLMs. Grounded in empirical human data, ActTraitBench establishes one-to-one mappings between psychometric facets and behavioral paradigms, and applies a Distributional Calibration via Quantile Mapping procedure to align LLM-judge score distributions with human norms. Experiments on 14 mainstream LLMs reveal a pervasive knowledge-decision asymmetry, where larger and more capable models often exhibit stronger behavioral divergence despite highly consistent self-reports. To mitigate this gap, we further introduce the Chain of Cognitive Alignment (CoCA), a plug-and-play inference-time intervention that improves alignment in reasoning-capable frontier models while exposing clear capability limitations in smaller architectures.
