Search papers, labs, and topics across Lattice.
This paper introduces GroupForward, a novel instance-grouped feed-forward Gaussian splatting model that reconstructs 3D environments from sparse, unposed, and uncalibrated multi-view images while incorporating explicit instance discrimination. By learning compact instance embeddings to group Gaussians into consistent 3D instances, GroupForward shifts the paradigm from per-Gaussian semantic feature rendering to instance-level semantic aggregation. The framework also includes a Referential Scene Reasoning Framework (RSRF) that enhances language interaction by enabling complex referential scene reasoning, significantly improving the accuracy of 3D referring segmentation tasks.
GroupForward transforms 3D scene understanding by enabling complex referential reasoning, moving beyond basic semantic queries to nuanced interactions with 3D instances.
Simultaneously reconstructing and understanding 3D environments is essential for embodied agents. Toward this goal, feed-forward semantic 3D Gaussian Splatting (3DGS) efficiently constructs semantic scene representations from sparse multi-view observations. However, existing methods lack explicit instance discrimination and mainly support category- or phrase-based semantic queries. To this end, we propose GroupForward, an instance-grouped feed-forward Gaussian splatting model that reconstructs geometry, appearance, instance structure, and semantics from sparse, unposed, and uncalibrated multi-view images. Unlike existing methods that attach high-dimensional semantic features to each Gaussian, GroupForward learns compact instance embeddings that group Gaussians into cross-view consistent 3D instances, reformulating feed-forward semantic 3DGS from per-Gaussian semantic feature rendering to instance-level semantic aggregation and propagation. Building on these instance groups, we further propose a Referential Scene Reasoning Framework (RSRF) for complex 3D referring segmentation. RSRF constructs an instance-grouped 3D scene graph and retrieves candidate instances for a given referring expression. A vision-language model then reasons over structured instance evidence and multi-view observations to identify the referred instance among the candidates. RSRF thereby extends language interaction from simple semantic querying to complex referential scene reasoning. Experiments on semantic reconstruction and referential reasoning demonstrate the effectiveness of our instance-grouped reconstruction and reasoning framework.
