Search papers, labs, and topics across Lattice.
This paper introduces MPIE-Bench, a benchmark designed to evaluate the anatomical plausibility of multi-person interactions in text-to-image editing, addressing significant failures in current models when generating images of multiple subjects engaged in contact actions. The authors highlight that existing evaluation methods overlook critical geometric issues, leading to high discrepancies between model performance and human judgment. Results show that while mesh evaluations for anatomy and interaction achieve scores of 0.65 and 0.72 respectively, traditional VLM checklists inaccurately rate the same images above 0.95, indicating a need for more nuanced evaluation metrics that align with human perception.
Current text-to-image models struggle with realistic multi-person interactions, scoring poorly on anatomical accuracy despite high VLM checklist ratings.
Text-to-image and personalized editing models now synthesize high-fidelity single-subject images with ease. Yet placing multiple named people into shared contact actions such as embrace, carry, or grapple still exposes major failures: fused limbs, invented extremities, and interpenetrating bodies. Existing evaluations largely overlook these anatomical and geometric issues, and VLM-as-a-judge checklists often saturate on Interaction while the errors remain obvious to humans. We introduce MPIE-Bench, a 2,500-sample benchmark of video-mined editing triplets spanning 405 scenes, 14 interaction categories, and four contact densities (C0-C3). We also propose MPIE-Eval, whose two new axes score contact-time geometry from a frozen public multi-person mesh reconstruction. Anatomy asks whether every human-like mass is explained by a complete set of reconstructed bodies, and Interaction asks whether the penetration and surface distance between those bodies match the contact the instruction asked for. Across ten editors, mesh Anatomy tops out at 0.65 and mesh Interaction at 0.72 on two different models, so no single editor is strong on both, while VLM checklists rate the same images above 0.95. A five-rater study confirms that both axes track human judgement more closely than a zero-shot VLM judge, and the rankings hold under ablation of every weight and threshold.
