Search papers, labs, and topics across Lattice.
This paper introduces evaluability-aware proposal planning for AI assistants, addressing the limitations of existing methods that assume users can reliably evaluate proposals despite bounded rationality. By formalizing this as a hidden-parameter sequential assistance problem (ProSE), the authors employ a KL-regularized binary response model to balance value gain against evaluability penalties. The results demonstrate that their approach, implemented as \textsc{ProSE-Plan}, significantly outperforms traditional methods in scenarios where evaluation costs are high, revealing the importance of user evaluability in AI assistance.
Proposals that consider user evaluability can dramatically enhance AI assistant performance, revealing that what users accept and what they need to learn can diverge significantly.
AI assistants often collaborate by proposing candidate edits, plans, or designs that users evaluate before adoption. Existing assistance methods focus on proposal quality or user-goal inference, often assuming that the user can reliably evaluate any proposal, which can fail in practice because of bounded rationality. We study evaluability-aware proposal planning, where proposals serve both as task interventions and as probes for learning latent preferences and evaluation constraints, where the resulting belief updates then guide later proposals. We formalise this setting as ProSE, a hidden-parameter sequential assistance problem, and instantiate it with a KL-regularised bounded-rational binary response model in which acceptance trades off value gain against a distance-dependent evaluability penalty. Analysing the planning consequence of this likelihood reveals that likely accepted proposals and informative probes need not coincide, which explains why planners that only pursue acceptance systematically underperform. We operationalise ProSE with \textsc{ProSE-Plan}, a depth-2 Bayes-adaptive planner that scores proposals by possible responses and response-induced posterior beliefs. In controlled graph simulations, \textsc{ProSE-Plan} improves over evaluability-unaware and myopic baselines when evaluation cost is the bottleneck, and a probe-commit ablation confirms that our approach selects informative proposals that simpler methods miss. Our results thus identify user evaluability as a planning-relevant dimension of AI assistance, complementary to generation quality and preference inference.
