Search papers, labs, and topics across Lattice.
The paper introduces PartRAG, a retrieval-augmented framework for single-image 3D generation with part-level structure, addressing limitations in prior-based methods and editability. PartRAG uses a Hierarchical Contrastive Retrieval module to align image patches with a part database, injecting diverse 3D part exemplars into a diffusion transformer's denoising process. The framework also incorporates a masked, part-level editor for localized modifications in a shared canonical space, achieving improved Chamfer Distance and F-Score on datasets like Objaverse, ShapeNet, and ABO.
Forget generating 3D objects from scratch – PartRAG lets you assemble them from a database of parts and edit them locally, like LEGOs for AI.
Single-image 3D generation with part-level structure remains challenging: learned priors struggle to cover the long tail of part geometries and maintain multi-view consistency, and existing systems provide limited support for precise, localized edits. We present PartRAG, a retrieval-augmented framework that integrates an external part database with a diffusion transformer to couple generation with an editable representation. To overcome the first challenge, we introduce a Hierarchical Contrastive Retrieval module that aligns dense image patches with 3D part latents at both part and object granularity, retrieving from a curated bank of 1,236 part-annotated assets to inject diverse, physically plausible exemplars into denoising. To overcome the second challenge, we add a masked, part-level editor that operates in a shared canonical space, enabling swaps, attribute refinements, and compositional updates without regenerating the whole object while preserving non-target parts and multi-view consistency. PartRAG achieves competitive results on Objaverse, ShapeNet, and ABO-reducing Chamfer Distance from 0.1726 to 0.1528 and raising F-Score from 0.7472 to 0.844 on Objaverse-with inference of 38s and interactive edits in 5-8s. Qualitatively, PartRAG produces sharper part boundaries, better thin-structure fidelity, and robust behavior on articulated objects. Code: https://github.com/AIGeeksGroup/PartRAG. Website: https://aigeeksgroup.github.io/PartRAG.
