Search papers, labs, and topics across Lattice.
This paper addresses the limitations of current image editing methods in handling complex Human-Object Interactions (HOI) by introducing HOI-Edit, a benchmark that evaluates dynamic interactions through an automated metric called HOI-Eval. The authors demonstrate that Image-to-Video (I2V) models are particularly effective for dynamic editing due to their temporal generation capabilities, which allow for a unique analysis of errors through a "replay of the failure process." By implementing the Self-Correcting Process Editing (SCPE) framework, they achieve competitive performance against state-of-the-art editing models while ensuring more accurate representations of target HOIs in generated videos.
I2V models not only excel at dynamic editing but also provide a unique lens for diagnosing errors in Human-Object Interaction tasks.
Current image editing methods excel at static attributes but fail at complex Human-Object Interactions (HOI), a critical challenge unaddressed by existing benchmarks that conflate HOI with static attributes, relying on global metrics incapable of simultaneously assessing dynamic interaction validity and entangled human-object pair preservation. Thus, we first introduce HOI-Edit, a comprehensive benchmark with three progressive cognitive levels, which features an automated metric HOI-Eval that reliably evaluates instance-level interaction by letting VLM Q&A after thinking with images containing grounded Human-Object pairs. Considering the task's essence of remodeling dynamic relationships, we benchmark Image-to-Video (I2V) models, finding them inherently suited for dynamic editing due to their temporal generation capabilities. Crucially, beyond superior performance, this capability provides a "replay of the failure process," offering unique diagnosability into why errors occur. We thus propose SCPE (Self-Correcting Process Editing), a novel, agentic self-correcting framework that constrains the generation of I2V models through iteratively refined prompts, enabling the generated videos to more accurately present the target HOI. Extracted frames from these videos are the final editing results. On HOI-Edit, SCPE achieves performance competitive with state-of-the-art (SOTA) editing models like Nano Banana on interaction. Code is available at https://github.com/oceanflowlab/HOI-Edit.
Tsinghua AI