Search papers, labs, and topics across Lattice.
This study investigates the self-consistency of numerical preference judgments derived from large language models (LLMs) by measuring how well these judgments align with a single utility function. Using statistical tests and interpretable measures, the authors found significant discrepancies in the willingness-to-pay responses across various contexts, including flights, apartments, and hotels. The results indicate that LLM-derived preferences are not reliably self-consistent, challenging the assumption that a unified utility function can accurately represent these judgments.
LLM-derived preference judgments reveal significant inconsistencies, undermining the reliability of using a single utility function for decision-making.
Agents increasingly interpret a person's natural-language preferences by querying an LLM for numerical preference judgments, e.g., by asking how much the person would be willing to pay for an item. A growing body of work estimates a utility function from these judgments and then chooses actions based on their estimated utility. This pipeline assumes the judgments are approximately self-consistent: that a single utility function can reproduce them. But are they? To study this question, we measure the self-consistency of cardinal LLM preference judgments. For example, the difference in stated willingness-to-pay between two items should match the stated payment that makes a person indifferent to exchanging them. We develop statistical tests and interpretable measures of how far observed responses depart from the best-fitting self-consistent utility function. Experiments with flight, apartment, and hotel examples across six LLMs reveal large persistent inconsistencies. This suggests that LLM-derived preference judgments cannot be faithfully summarized by a single utility function.
