Search papers, labs, and topics across Lattice.
Pair2Scene generates high-fidelity 3D indoor scenes by learning local inter-object relations, specifically support and functional dependencies, using a novel network trained on a curated 3D-Pairs dataset. The network estimates spatial position distributions of dependent objects conditioned on anchor objects' position and geometry. This approach enables the generation of complex, physically and semantically plausible scenes by recursively applying the learned rules within a hierarchical structure, outperforming existing methods in generalization and scene density.
Forget LLMs – generating realistic 3D scenes might just hinge on learning how objects relate to each other locally.
Generating high-fidelity 3D indoor scenes remains a significant challenge due to data scarcity and the complexity of modeling intricate spatial relations. Current methods often struggle to scale beyond training distribution to dense scenes or rely on LLMs/VLMs that lack the ability for precise spatial reasoning. Building on top of the observation that object placement relies mainly on local dependencies instead of information-redundant global distributions, in this paper, we propose Pair2Scene, a novel procedural generation framework that integrates learned local rules with scene hierarchies and physics-based algorithms. These rules mainly capture two types of inter-object relations, namely support relations that follow physical hierarchies, and functional relations that reflect semantic links. We model these rules through a network, which estimates spatial position distributions of dependent objects conditioned on position and geometry of the anchor ones. Accordingly, we curate a dataset 3D-Pairs from existing scene data to train the model. During inference, our framework can generate scenes by recursively applying our model within a hierarchical structure, leveraging collision-aware rejection sampling to align local rules into coherent global layouts. Extensive experiments demonstrate that our framework outperforms existing methods in generating complex environments that go beyond training data while maintaining physical and semantic plausibility.
