Search papers, labs, and topics across Lattice.
The authors formalize the task of conversational revision propagation—updating all dependent elements within an artifact when a user specifies only a localized edit—and establish an empirical benchmark across several open-weight and proprietary LLMs. Maintaining artifact consistency is a critical failure mode in interactive generation, as structural dependencies are often implicitly distributed across extended dialogue history. Evaluating nine test-time compute strategies shows that selecting from just three parallel samples via medoid or LLM-based reranking achieves the best cost-performance Pareto frontier, improving accuracy by 2.2–9.7% over baseline levels.
Propagating local edits across multi-turn artifacts does not require expensive sequential reflection loops—reranking just three parallel samples reliably yields up to a 9.7% consistency boost across open and frontier models.
Large Language Models (LLMs) often help users generate artifacts through iterative cycles of generation and revision in conversation. A challenge here is that, when users specify only a local change during revision, LLMs must instead identify the relevant dependencies and propagate the revision to all affected parts of the artifact. This paper studies this ability of LLMs on conversationally generated artifacts, where the artifact context and its dependencies may be embedded in the conversation history. Toward practical use, we also explore cost-effective test-time compute for this new setting. Specifically, we introduce a new benchmark for this setting, and evaluate nine revision methods, including sequential reflection and parallel sampling variants, using gpt-oss-20b/120b, gpt-5.4-mini, and qwen3.5-9b/27b/122b on the benchmark. The results show that baselines achieve accuracies of 68.3--93%, and the most cost-effective method is selecting from three parallel samples using either LLM-based or medoid selection, which improves accuracy by 2.2--9.7%. Our code and dataset are available at https://github.com/ntt-dkiku/llm-revision-propagation.
