Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces the concept of representational accuracy as a metric for evaluating how well AI systems capture user interpretations, operationalized through a Behavioral Specification that compresses personal data into interpretable patterns for language models. The authors demonstrate that this approach significantly enhances representational accuracy while reducing context costs by approximately 25 times, effectively minimizing model hedging across various context conditions. Notably, the method is particularly effective for interpretation-required questions, indicating that accurate user representation is crucial for aligning AI behavior with user intent, especially for underrepresented populations in pretraining datasets.
Achieving 25x less context cost while nearly eliminating model hedging reveals a transformative approach to aligning AI with user intent through enhanced representational accuracy.
If an AI agent makes decisions on a person's behalf, those decisions must align with its user. We introduce representational accuracy to measure how faithfully a system captures a person's interpretation. An interpretive layer is operationalized as a Behavioral Specification. Our reference implementation aggressively compresses a person's data into interpretive patterns, served as context to a language model. We evaluate the Specification on a prototype benchmark of held-out behavioral predictions scored by a calibrated 5-judge LLM panel. We test it independently and in composition with a range of context conditions: full raw corpus, full extracted facts, and four commercial memory systems (Mem0, Letta, Supermemory, Zep). Across 14 public-domain autobiographical corpora, the Specification lifts representational accuracy in aggregate and nearly eliminates model hedging. It recovers most of what the raw corpus delivers, at ~25x less context cost. The Specification lifts subjects toward a common predictive level regardless of pretraining baseline; the lift in absolute points is therefore largest where the baseline is lowest, suggesting the population of relevance is anyone not adequately represented in pretraining. Lift is greatest on interpretation-required questions, where providing an interpretive layer enables model behavior that extracted facts or raw corpus do not. Conversely, on recall-required questions, this layer can interfere rather than help. We conclude that representational accuracy is distinct from recall and that human-AI alignment is dependent on how accurately the user is represented. Representational accuracy makes that alignment testable.