Search papers, labs, and topics across Lattice.
This paper introduces DRFLOW, a benchmark designed to evaluate the ability of agents to predict personalized workflows from diverse information sources, addressing a gap in existing deep research systems that primarily focus on report generation. The benchmark includes 100 tasks across five domains, with a total of 1,246 reference workflow steps sourced from over 3,900 documents, and employs seven diagnostic metrics to assess performance. Results show that the proposed DRFLOW-Agent (DRFA) outperforms strong baseline agents by up to 10.02% in average F1 score, yet highlights the ongoing challenges in achieving accurate and complete workflow predictions.
Agents still struggle to accurately predict personalized workflows, with current models showing significant room for improvement despite promising advancements.
Deep research (DR) systems are increasingly used for complex information-seeking tasks, but existing works mainly focus on generating reports and summaries. In contrast, many enterprise tasks instead require an agent to identify concrete workflows which is a sequence of action-steps. For example, rather than summarizing budgeting policies, an agent should be able to determine the steps needed to answer a question such as: "How do I request new headcount given a fixed budget?". Therefore, we introduce DRFLOW, a benchmark for evaluating personalized workflows predicted by agents from heterogeneous sources. Each task requires the agent to identify relevant evidence from scattered sources, then use that evidence to predict the correct action-step sequence for the user's task. DRFLOW contains 100 tasks across five domains, with 1,246 reference workflow steps grounded in more than 3,900 sources. We define seven diagnostic metrics covering factual grounding, step recovery, structural ordering, condition resolution, and personalization. We further present DRFLOW-Agent (DRFA), a workflow-oriented reference agent to predict personalized workflow. We show that although DRFA improves over strong baseline agents (upto 10.02% average F1 score), there is substantial room for improvement remains across these workflow metrics, indicating that predicting complete and correct personalized workflows remains a challenging frontier for deep research.
