Search papers, labs, and topics across Lattice.
The paper introduces OVOW, a novel training-free system that reconstructs instance-level, simulation-ready 4D mesh scenes from a single monocular video. By employing a four-stage pipeline that includes vision-language instance classification, category-aware reconstruction, iterative optimization for pose recovery, and physics-grounded assembly, OVOW produces watertight meshes suitable for physics simulations. The system outperforms existing methods in accuracy and speed, achieving the best overall layout and geometry accuracy across benchmarks while significantly enhancing the efficiency of video-to-4D scene reconstruction.
OVOW achieves unprecedented accuracy and speed in reconstructing 4D scenes from a single video, making it a game-changer for physics simulation in AI.
We introduce \textbf{OVOW}, the first training-free system that reconstructs \emph{instance-level, simulation-ready} 4D mesh scenes from a single monocular video. Recent 4D reconstruction achieves impressive rendering quality, but its outputs (\eg, implicit fields, Gaussian primitives, or point clouds) lack the watertight topology, instance separation, and standardized physical interfaces required by physics simulators and embodied AI. OVOW closes this gap with a four-stage pipeline: a vision-language model discovers, labels, and motion-classifies all instances; category-aware reconstruction yields per-instance meshes for rigid objects and topology-consistent mesh sequences for deformable ones; an iterative render-match-optimize procedure recovers metric scale and 6-DoF pose trajectories; and physics-grounded assembly enforces ground contact and inter-object support. Crucially, we model all motion, rigid and non-rigid, through direct vertex deformation without category-specific priors or skeleton rigging, producing watertight mesh scenes ready for downstream physics simulation and editing. We further establish the first benchmark for \emph{structured Video-to-4D} evaluation, with metrics for geometric correctness, instance separation, and physical plausibility beyond visual fidelity; the same pipeline doubles as a scalable engine for \emph{synthesizing} paired video-to-4D simulation data for future 4D world models and embodied AI. Across two synthetic benchmarks (static and 4D), OVOW attains the best overall layout and geometry accuracy and the lowest photometric and semantic error among all baselines, and on monocular video runs one to two orders of magnitude faster than the baselines, while downstream physics simulation confirms its physical stability.
Tsinghua AI