Search papers, labs, and topics across Lattice.
Tsinghua AIAgricultural UniversityNational Supercomputing CenterNational Supercomputing Center in ShenzhenNortheasternSouthwest Jiaotong UniversitySouthwest USYSUMay 21, 2026arXiv:2605.22366AgroTools is introduced as a new benchmark for evaluating tool-augmented multimodal agents in agriculture, featuring 539 question-answer instances and 1,097 agricultural images across five task families. The benchmark includes an executable environment with 14 agricultural tools and provides structured tool-use traces for evaluating both process-level execution and outcome-level success. Benchmarking of 13 open- and closed-source multimodal LLMs reveals significant limitations in tool planning, argument generation, execution recovery, and final-answer synthesis within agricultural contexts.
Today's best multimodal LLMs are surprisingly inept at using tools to solve agricultural tasks, struggling with everything from planning to error recovery.
Agricultural decision-making increasingly requires multimodal systems that can transform visual observations into reliable, executable actions. However, existing agricultural multimodal benchmarks mainly evaluate final-answer correctness and provide limited support for assessing whether models can use external tools to complete precision-sensitive workflows. In this paper, we introduce AgroTools, a benchmark for evaluating tool-augmented multimodal agents in agriculture. AgroTools contains 539 question-answer instances paired with 1,097 heterogeneous agricultural images, spanning five task families and an executable environment of 14 agricultural tools. Each query is annotated with structured tool-use traces, enabling a dual-view evaluation of both process-level execution quality and outcome-level task success. We benchmark 9 open-source and 4 closed-source multimodal large language models on AgroTools. Results show that current models remain far from reliable in agricultural tool-use settings, with clear bottlenecks in tool planning, argument generation, execution recovery, and final-answer synthesis. We hope AgroTools will support future research on multimodal agents for high-precision agricultural applications. The benchmark and evaluation are available at https://huggingface.co/datasets/AgroTools/AgroTools.
