Search papers, labs, and topics across Lattice.
The ROAD framework leverages discriminative 3D foundation models to enhance the efficiency of high-fidelity 3D shape generation by transferring semantic and structural priors into diffusion transformers. By employing a reciprocal-objective alignment strategy, it ensures both global semantic coherence and precise alignment of geometric details, addressing the challenges posed by the differences in latent representations. This approach achieves competitive performance with only 1.5% of the training data compared to the industrial baseline Step1X-3D, significantly lowering the computational costs associated with 3D generation.
Achieving high-fidelity 3D generation with just 1.5% of the training data could revolutionize resource allocation in 3D modeling.
High-fidelity 3D generation predominantly relies on scaling model capacity and data, which incurs prohibitive computational costs. This paradigm typically requires learning geometry from scratch and overlooks the rich semantic and structural priors already encapsulated in discriminative 3D foundation models. We contend that leveraging the profound understanding of the 3D world possessed by these discriminative models can significantly reduce generative cost. To this end, we propose ROAD, a framework that reduces the training cost of 3D generation by transferring these rich discriminative priors into diffusion transformers. To address the inherent semantic-structural heterogeneity between generative and discriminative latents, we introduce a reciprocal-objective alignment strategy. This method synergizes Holistic Semantic Condensing to enforce global semantic coherence and Structural Optimal Alignment, which is formulated as a bipartite matching problem to rigorously align microscopic geometric details between disparate latent spaces. The 3D foundation model is only used for training-time supervision of alignment and is not used at inference, incurring no additional inference cost. Compared with the industrial baseline Step1X-3D, the proposed ROAD achieves highly competitive generation performance with only 1.5% of the training data and significantly reduces training costs, effectively reducing the computational overhead of high-fidelity 3D generation. Code is available at https://github.com/H-EmbodVis/ROAD.
