Search papers, labs, and topics across Lattice.
This paper introduces a novel task of personalized ambiguity adaptation in coding assistants, which leverages resolved session history to address recurring ambiguities in user requests across sessions. By benchmarking this task with the CAPA framework, which includes 600 coding sessions and evaluates 12 recent LLMs, the authors demonstrate that using same-user history significantly improves executable success and reduces the need for clarifications. The key finding reveals that incorporating user-specific memory can enhance coding assistant performance, aligning generated code more closely with user intent while minimizing repeated interactions.
Leveraging user history can cut clarification requests in coding assistants by identifying and resolving recurring ambiguities, leading to more efficient coding sessions.
AI-assisted coding increasingly translates informal user intent into executable software, yet coding requests often contain ambiguities that recur in user-specific ways across tasks and sessions. Existing disambiguation methods typically address each ambiguous request in isolation within the current coding session, often through eliciting additional clarification. However, whether resolved session history from the same user can serve as memory for resolving recurring personalized ambiguity in a newly opened session remains underexplored. We formulate personalized ambiguity adaptation as a new task: given a user's previously resolved coding sessions and a new ambiguous request, an assistant should identify the recurring ambiguity pattern, produce the intended executable solution, and minimize clarification. To benchmark this task, we introduce CAPA, which characterizes personalized coding ambiguity through six mechanisms and injects these mechanisms into unambiguous executable tasks using a controlled three-stage generation pipeline. CAPA contains 600 coding sessions across 60 balanced user--ambiguity cells, including 300 held-out evaluation sessions. We evaluate 12 recent LLMs under no-history and same-user-history conditions using executable success, first-turn success, and turns-to-completion. Our analyses examine task difficulty, user identity, and memory-based history use, and we further propose same-user history gating as a lightweight inference-time method. CAPA provides a foundation for developing long-term coding assistants that better align generated code with user intent while reducing repeated clarification.
