Search papers, labs, and topics across Lattice.
This paper introduces TOFFEE, a novel system that synthesizes high-quality data agent trajectories using Monte Carlo Tree Search (MCTS) combined with adaptive model selection and cross-task prefix reuse. The motivation behind TOFFEE is to enhance the generalization capabilities of LLM-powered data agents in heterogeneous enterprise environments, addressing the limitations of existing systems that struggle with unseen data. Key results show that TOFFEE effectively generates scalable trajectory data, facilitating both supervised finetuning and in-context learning for complex analytical tasks across diverse settings.
TOFFEE can synthesize high-quality data agent trajectories that significantly improve LLM performance in unfamiliar analytical workflows.
LLM-powered data agents are playing an increasingly important role in data-driven decision making. However, existing data agents struggle to generalize to unseen data environments and analytical workflows, especially in heterogeneous enterprise settings. This creates a growing need for synthesizing high-quality data agent trajectories that capture complex analytical workflows for given data environments. Such trajectories support two key downstream uses: they can serve as supervised finetuning (SFT) data that adapts data agent models to the target domain, and as in-context learning (ICL) demonstrations to guide general-purpose LLMs in unfamiliar data environments. Thus, we introduce TOFFEE, a system for synthesizing high-quality data agent trajectories from given data environments via Monte Carlo Tree Search (MCTS) with adaptive model selection and cross-task prefix reuse. We show that TOFFEE can effectively generate scalable trajectory data for complex analytical tasks across heterogeneous environments. In this demonstration, we present the system framework of TOFFEE, including its task pool construction, trajectory explorer, and learned cost model. We also introduce the web interface of TOFFEE and its workflow, and demonstrate two end-to-end scenarios: trajectory synthesis for data agent finetuning, and demonstration-augmented data agent reasoning.
