Search papers, labs, and topics across Lattice.
This paper introduces PhysAgent, a novel multi-agent framework that automates the synthesis of physically plausible 4D motion by integrating a simulator-in-the-loop approach with multimodal inputs. By decoupling intrinsic material properties from extrinsic dynamics, PhysAgent employs a Semantic Agent and Refine Agents that utilize trajectory-grounded feedback to optimize force field configurations dynamically. Experimental results show that PhysAgent not only enhances generation diversity but also significantly improves physical accuracy compared to existing methods, addressing critical bottlenecks in large-scale simulation data generation.
PhysAgent achieves unprecedented levels of physical accuracy and diversity in 4D motion synthesis by leveraging trajectory-grounded feedback from multiple agents.
Achieving fully automated, physically plausible 3D motion synthesis is a core objective in graphics and generative AI. However, configuring complex environmental force fields still relies entirely on manual expert intervention, creating a severe bottleneck for large-scale simulation data generation. Existing automated methods primarily focus on material optimization and exhibit severe modality gaps and technical flaws when applied to the vastly more complex force field optimization space: naive Large Language Models (LLMs) lack underlying simulation feedback, causing severe physical inaccuracies, while traditional Score Distillation Sampling (SDS) suffers from sluggish gradients, local optima entrapment, and a mathematical inability to dynamically switch discrete force fields. To address this, we propose PhysAgent, the first simulator-in-the-loop multi-agent framework that leverages multimodal inputs for automated, physically grounded 4D synthesis. By decoupling intrinsic materials from extrinsic dynamics, PhysAgent utilizes a Semantic Agent equipped with an externalized Force Field Skill module to master simulation rules and generate valid initializations. Subsequently, the Refine Agents, driven by Trajectory-Grounded Multi-Agent Feedback, leverage vision foundation models to extract dense point trajectories from rendered frames. By converting these explicit motion trajectories into structured textual descriptors, the agent harnesses LLM commonsense reasoning to execute zero-shot macroscopic leaps, effectively escaping local optima and dynamically switching discrete force fields. Extensive experiments demonstrate that PhysAgent rapidly generates stable, diverse physical scenes from arbitrary multimodal prompts, significantly outperforming existing baselines in both generation diversity and physical accuracy.
