Search papers, labs, and topics across Lattice.
The paper introduces Code2Worlds, a framework for generating 4D dynamic scenes by formulating the task as language-to-simulation code generation. It addresses the challenges of multi-scale context entanglement and the semantic-physical execution gap using a dual-stream architecture for object generation and environmental orchestration, coupled with a physics-aware closed-loop mechanism for dynamic fidelity. Experiments on the Code4D benchmark demonstrate that Code2Worlds significantly outperforms existing methods, achieving a 41% SGS gain and 49% higher Richness by generating physics-aware dynamics.
Coding LLMs can now generate more physically plausible and dynamically rich 4D worlds, thanks to a novel closed-loop framework that iteratively refines simulation code based on physics-aware self-reflection.
Achieving spatial intelligence requires moving beyond visual plausibility to build world simulators grounded in physical laws. While coding LLMs have advanced static 3D scene generation, extending this paradigm to 4D dynamics remains a critical frontier. This task presents two fundamental challenges: multi-scale context entanglement, where monolithic generation fails to balance local object structures with global environmental layouts; and a semantic-physical execution gap, where open-loop code generation leads to physical hallucinations lacking dynamic fidelity. We introduce Code2Worlds, a framework that formulates 4D generation as language-to-simulation code generation. First, we propose a dual-stream architecture that disentangles retrieval-augmented object generation from hierarchical environmental orchestration. Second, to ensure dynamic fidelity, we establish a physics-aware closed-loop mechanism in which a PostProcess Agent scripts dynamics, coupled with a VLM-Motion Critic that performs self-reflection to iteratively refine simulation code. Evaluations on the Code4D benchmark show Code2Worlds outperforms baselines with a 41% SGS gain and 49% higher Richness, while uniquely generating physics-aware dynamics absent in prior static methods. Code: https://github.com/AIGeeksGroup/Code2Worlds. Website: https://aigeeksgroup.github.io/Code2Worlds.
