Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces ScenePilot, a retrieval-augmented Grow-and-Repair framework for text-driven 3D indoor scene generation that addresses limitations of existing methods by incrementally generating scenes with learned rectification. The approach utilizes a Hierarchical Retrieval-Augmented Planning (HRAP) module to retrieve layout priors and a Reinforcement Multimodal Repair (RMR) module for local and global corrections, ensuring geometrically valid and functionally coherent layouts. The authors validate their method using the SceneReverse-17k dataset, demonstrating significant improvements in physical plausibility and controllability compared to traditional one-pass generators and heavy optimization techniques.
ScenePilot achieves superior scene generation by combining incremental growth with intelligent rectification, resulting in more physically plausible and functionally coherent 3D environments.
Text-driven 3D indoor scene generation has advanced from dataset-bound layout modeling to open-vocabulary synthesis with large language and vision-language models. Yet existing methods remain limited: one-pass generators often yield geometrically invalid layouts, heavy post-hoc optimization is costly and unstable, and prompt-only planners lack reusable layout priors for functional grouping and object relations. We propose \textbf{ScenePilot}, a retrieval-augmented \textbf{Grow-and-Repair} framework that formulates scene generation as prior-guided incremental growth with learned rectification. Given a prompt, the Hierarchical Retrieval-Augmented Planning (HRAP) module retrieves room-, group-, and anchor-level layout priors to support functional group planning. A text-driven base generator then inserts object groups sequentially, while the Reinforcement Multimodal Repair (RMR) module performs lightweight local correction after each insertion and a final global repair after completion. To train this policy, we construct \textbf{SceneReverse-17k}, a repair-trajectory dataset built by perturbing high-quality 3D scenes in position, rotation, and scale, then using inverse operations as executable rectification targets. The policy predicts structured \emph{move--rotate--scale} actions from rendered views, scene state, retrieved priors, and edit history. By combining HRAP with RMR, ScenePilot offers an efficient alternative to one-shot generation and heavy full-scene optimization, improving physical plausibility, functional coherence, and controllability while preserving diversity.