Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This study reveals that a single policy model can outperform hierarchical evolutionary Monte Carlo Tree Search (MCTS) in the context of chemistry tool learning, addressing the challenges of tool selection, argument filling, and call chaining. By interleaving reasoning and tool calls in a left-to-right generation process, the proposed model eliminates the need for learned critics or judges during training, relying instead on supervised warm-up and outcome-level reinforcement learning. The results demonstrate significant improvements in Tool F1 and Return F1 metrics on ChemToolBench, showcasing the efficiency of this approach compared to traditional search methods.
A single policy model outperforms complex tree search methods in chemistry tool learning, achieving notable improvements in performance metrics.
Chemistry questions often demand exact computation and database lookups that a language model cannot supply from its parameters, so it must reach for external tools. Tool use here is a three-part problem: select the right tool from a large pool, fill it with correctly typed arguments, and chain calls so that each consumes the outputs of the last. CheMatAgent, a previously published system, addresses this with hierarchical evolutionary MCTS: separate policy and execution models searching tool-call trees under two learned critics, one regressed partly onto GPT-assigned scores. We show that a single policy suffices. Our model interleaves reasoning, tool calls, and returns in one left-to-right generation, trained by a supervised warm-up and then outcome-level reinforcement learning against a programmatic reward read directly off the gold call chain, which leaves no learned critic and no judge in the training loop. On ChemToolBench multiple-tool comprehensive chemistry, on both backbones CheMatAgent use, we improve Tool F1 by 5.5% and Return F1 by 9.6% on Qwen-2.5-7B, and by 3.7% and 3.9% on Llama-3.1-8B, compared with their strongest search configuration, at one model invocation per question, against a search whose cost grows with the tree; we also lead answer Pass Rate on Qwen-2.5-7B.
Mila