Search papers, labs, and topics across Lattice.
SPARGen introduces a unified multimodal framework that integrates spatial perception and reasoning by treating 3D reconstruction, dense correspondence, and spatial reasoning as instruction-conditioned generation tasks. This approach allows for the serialization of structured and linguistic outputs into token sequences while generating dense geometric fields aligned with images, facilitating shared representation learning. Experimental results demonstrate that SPARGen outperforms existing methods across various benchmarks, highlighting its effectiveness in addressing multiple spatial tasks within a single model.
SPARGen achieves competitive performance across diverse spatial tasks by unifying perception and reasoning into a single generative framework, challenging the need for separate architectures.
Spatial perception and reasoning from visual observations require recovering geometric structure, establishing correspondences, and understanding spatial relations. Existing approaches typically address these capabilities separately using task-specific architectures or external geometric modules, limiting knowledge transfer among complementary representations of the same physical scene. We introduce SPARGen, a unified multimodal framework that casts 3D reconstruction, dense correspondence, and spatial reasoning as instruction-conditioned generation tasks. SPARGen serializes compact structured and linguistic outputs as token sequences while generating dense geometric fields in image-aligned forms, enabling spatial supervision to jointly shape shared representations within a native multimodal generative model. Experiments across benchmarks for 3D reconstruction, correspondence, and spatial reasoning show that SPARGen achieves competitive performance across heterogeneous spatial tasks within a single native multimodal generative framework.
