Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
To eliminate the high inference latency of autoregressive multimodal LLMs in tool-based image editing, the authors formulate continuous tool parameter estimation as a conditional rectified flow problem. The resulting framework, FlowTool, pairs a vision-language backbone for instruction understanding with a Diffusion Transformer that generates continuous retouching parameters directly from Gaussian noise. Across benchmarks like MIT-Adobe5K and MMArt-Bench, FlowTool outperforms specialized editing agents and proprietary MLLMs while slashing inference latency by at least 50× and halving memory consumption.
Ditching autoregressive token generation in favor of continuous flow matching slashes tool-use latency by over 50× while beating proprietary MLLMs at image retouching.
Tool-based image editing (image retouching) is commonly formulated with autoregressive multimodal large language models (MLLMs) that sequentially generate reasoning, tool selections, and parameter values. In this work, we present a novel approach to tool-based image editing by framing the task as a flow matching problem. We introduce FlowTool, a framework that directly models the distribution of high-quality tool parameters conditioned on the input image and user instruction using conditional rectified flow. FlowTool combines a vision-language model backbone for multimodal understanding with a Diffusion Transformer parameter generator that transforms Gaussian noise into an editing plan. We train FlowTool with a two-stage supervised flow-matching curriculum, followed by reward-based post-training. Across MMArt-Bench, FlowTool-Eval, ArtEdit-Bench, and MIT-Adobe5K, FlowTool achieves significantly stronger reference-based performance than specialized MLLM editing agents and proprietary MLLMs, while remaining competitive with proprietary models under reference-free evaluation. Moreover, FlowTool significantly improves inference efficiency, reducing latency by at least $50\times$ while requiring nearly $2\times$ less memory than the compared baselines. These results demonstrate that tool-based image editing can be effectively modeled as conditional generation over structured continuous editing parameters, without autoregressive reasoning.