Track synthesis, materials and catalysis as it is published.
Search papers, labs, and topics across Lattice.
The authors present SolveEdit, a 2,728-case benchmark that evaluates whether generative models can infer implicit, goal-driven visual transformations from scenes or rules and execute them without disturbing unrelated context. To evaluate open-ended visual reasoning without relying on static reference images, the benchmark uses SolveScore to verify atomic transition contracts that check both required goal states and protected invariant regions. Frontier models struggle with visual problem solving—topping out at a 57.0% SolveScore—though decomposing the task into an explicit visual planning stage before generation boosts GPT-Image-2 to 71.6% without modifying the underlying editor.
Leading image generation models fail over 40% of goal-driven visual reasoning tasks, but decoupling explicit visual planning from generation boosts task completion by up to 14.6 points without fine-tuning.
Machine intelligence is often evaluated through abstract reasoning problems, yet many real-world problems are visual, such as arranging objects, repairing layouts, or tracing routes. Solving these problems requires understanding a scene, inferring what must change to achieve a goal, and realizing that change without disturbing unrelated content. However, existing benchmarks mainly evaluate perception, generation, or explicitly specified transformations, leaving goal-driven visual problem solving underexplored. To bridge this gap, we introduce SolveEpIT, a benchmark for visual problem solving through scene transformation. Given an image and a goal, a model must infer a valid transformation from the request, the scene, or a visually expressed rule, then execute it while preserving unrelated content. SoLvEEDrr contains 2,728 cases. Atomic transition contracts specify required and protected conditions, enabling SoLvEScoRE to measure completion and unintended changes without a single reference output. The strongest evaluated model achieves only57.0% SolvEScore. We further introduce SolveEdiT-PLAN, a two-stage visual planner that instantiates the transition before generation. Under matched single-generation evaluation, it improves SoLvEScoRE by 9.1 points on average across three tested generators, including a gain from 57.0% to 71.6% for GPT-Image-2, without modifying the editor.