Search papers, labs, and topics across Lattice.
This paper introduces TRACE-Bench, a novel benchmark for evaluating multi-reference image generation by formalizing four atomic operators that underpin diverse tasks in this domain. By structuring prompts as compositional formulas, the authors provide a systematic way to assess model capabilities and diagnose failures, revealing that the main limitations in current models stem from disentanglement and attribute binding rather than scene composition. Evaluation of nine leading models using this framework shows that even the top-performing model achieves only 0.74 on attribute fidelity, highlighting significant areas for improvement.
Disentanglement and attribute binding are the real bottlenecks in multi-reference image generation, not scene composition, with top models still struggling to achieve high fidelity.
Despite recent advances in unified multimodal models for multi-reference image generation, existing benchmarks remain organized around predefined task types (e.g.,"subject composition"), which are ill-suited to this combinatorial setting and lead to fragmented coverage, uncontrolled complexity, and little diagnostic value. Recognizing that diverse multi-reference tasks share a common set of atomic operations, we adopt a capability-oriented perspective and formalize four operators: Anchor ($f$), Disentangle ($g$), Apply ($\oplus$), and Compose ($C$). Any multi-reference prompt can then be represented as a compositional formula over these operators, whose structural complexity is quantified by the number of operator slots. Building on this formulation, we construct TRACE-Bench, comprising approximately 1,600 evaluation cases across slot counts 1--8, built from 631 formula templates and around 4,000 reference images spanning diverse artistic styles and real-world subjects. The formula structure directly drives an operator-aligned evaluation protocol for per-capability scoring and a diagnostic tree analysis for recursive failure localization. Evaluating 9 leading models reveals insights invisible to holistic scoring: the primary bottleneck lies in disentanglement ($g$) and attribute binding ($\oplus$) rather than scene-level composition ($C$), with even the best model scoring only 0.74 on attribute fidelity. Project page: https://amuseum-whr.github.io/TraceBench
