Search papers, labs, and topics across Lattice.
This study evaluates the performance of deep research agents (DRAs) across multiple turns of feedback, contrasting self-reflection with process-level feedback. The authors introduce Research Gap Inference (RGI) to identify gaps in research strategies based on rubric criteria, revealing that while self-reflection yields minimal improvement, a single round of process-level feedback can significantly enhance performance. However, the findings indicate that subsequent revisions often lead to regression on previously met criteria, highlighting the challenges in achieving consistent multi-turn improvements in DRAs.
A single round of targeted feedback can boost DRA performance by up to 15 points, but subsequent revisions may undo prior gains.
Existing benchmarks for deep research agents (DRAs) assess only single-shot outputs, ignoring a key question: can DRAs improve their reports when guided by feedback? To investigate this, we conduct a multi-turn evaluation of DRAs under two feedback settings: self-reflection, in which the agent revises its report without any external diagnostic signal, and process-level feedback, in which the agent receives guidance targeting gaps in its research strategy. To enable process-level feedback, we design Research Gap Inference (RGI), a method that analyzes patterns of satisfied and unsatisfied rubric criteria to infer research-process gaps. Our analysis reveals three key findings: (i) under self-reflection, agents incorporate and regress on rubric criteria at nearly equal rates, yielding negligible net improvement; (ii) a single round of process-level feedback yields substantial gains, raising the normalized score by approximately $8$-$15$ points and yielding a roughly $35$-$40\%$ incorporation rate; (iii) these gains do not compound over subsequent turns, as agents regress on up to $24\%$ of previously satisfied criteria when rewriting the full report to address remaining gaps. Even with targeted guidance, reliable multi-turn improvement remains out of reach for the DRA architectures we evaluate. Our code and results are publicly available at https://github.com/sabharwalrishabh/Multi-Turn-Evaluation-of-DRAs.
