Search papers, labs, and topics across Lattice.
This paper introduces CGGS, a novel text-to-3D framework that enhances ego-centric 3D scene generation by addressing the challenges of limited view overlap and perspective distortion. The approach involves fine-tuning a Multi-View Latent Diffusion Model with a consistency-augmented loss to produce high-fidelity 2D content, followed by a Layout Decorator that utilizes optical flow for depth estimation, and a Geometric Refiner that employs a Mutual Information Depth Loss for improved 3D reconstruction. Experimental results show that CGGS significantly outperforms existing methods in generating coherent and semantically aligned 3D scenes from textual descriptions.
CGGS achieves unprecedented coherence and accuracy in text-driven 3D scene generation, setting a new standard for ego-centric visual content creation.
Challenges remain in ego-centric 3D scene generation due to limited view overlap and the dominant influence of individual perspectives on scene interpretation. These factors hinder the creation of viewpoint-consistent and semantically aligned visual content, as well as the construction of accurate geometric structures. In this paper, we propose CGGS, a text-to-3D framework aiming to enhance 3D-content-awareness and address geometric distortions in ego-centric scene generation. Firstly, the Ego-centric Generator is proposed by fine-tuning a Multi-View Latent Diffusion Model with consistency-augmented loss to generate consistent, high-fidelity 2D content aligned with textual descriptions. Then, Layout Decorator leverages optical flow and point-track correspondence to estimate depth, therefore producing dense point clouds as coarse layouts from the ego-centric 2D priors. Building on this initialization, Geometric Refiner is proposed to enhance 3D Gaussian reconstruction via an entropy-based Mutual Information Depth Loss (MID) combined with a hierarchical optimization scheme for improving visual quality and geometric structure. Comprehensive experiments demonstrate that CGGS outperforms previous methods in generating coherent and accurate text-driven 3D scenes. Project page: [https://cggs-26.github.io/cggs26/](https://cggs-26.github.io/cggs26/).
