Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
SceneOrchestra introduces a trainable orchestration framework for 3D scene synthesis that optimizes tool-call execution flow and eliminates the step-by-step review loop. It employs an orchestrator-discriminator architecture, fine-tuned with a two-phase training strategy, to learn context-aware tool selection and full tool-call trajectory generation. Experiments demonstrate that SceneOrchestra achieves state-of-the-art scene quality with reduced runtime compared to existing agentic 3D scene synthesis methods.
Ditch the slow, iterative "execute-review-reflect" loop: SceneOrchestra's end-to-end trainable orchestrator slashes runtime and boosts 3D scene quality by predicting entire tool-call trajectories at once.
Recent agentic frameworks for 3D scene synthesis have advanced realism and diversity by integrating heterogeneous generation and editing tools. These tools are organized into workflows orchestrated by an off-the-shelf LLM. Current approaches typically adopt an execute-review-reflect loop: at each step, the orchestrator executes a tool, renders intermediate results for review, and then decides on the tool and its parameters for the next step. However, this design has two key limitations. First, next-step tool selection and parameter configuration are driven by heuristic rules, which can lead to suboptimal execution flows, unnecessary tool invocations, degraded output quality, and increased runtime. Second, rendering and reviewing intermediate results after each step introduces additional latency. To address these issues, we propose SceneOrchestra, a trainable orchestration framework that optimizes the tool-call execution flow and eliminates the step-by-step review loop, improving both efficiency and output quality. SceneOrchestra consists of an orchestrator and a discriminator, which we fine-tune with a two-phase training strategy. In the first phase, the orchestrator learns context-aware tool selection and complete tool-call trajectory generation, while the discriminator is trained to assess the quality of full trajectories, enabling it to select the best trajectory from multiple candidates. In the second phase, we perform interleaved training, where the discriminator adapts to the orchestrator's evolving trajectory distribution and distills its discriminative capability back into the orchestrator. At inference, we only use the orchestrator to generate and execute full tool-call trajectories from instructions, without requiring the discriminator. Extensive experiments show that our method achieves state-of-the-art scene quality while reducing runtime compared to previous work.