Search papers, labs, and topics across Lattice.
This paper introduces SeqAlign3DVG, a benchmark designed to enhance 3D visual grounding by ensuring strict alignment between text and observations while maintaining temporal order. The benchmark consists of over 24,000 samples with human-verified descriptions that address the complexities of multi-instance ambiguities and relationships. Utilizing a novel voxel-based pipeline with Relevance-Ordered Voxel Memory and Progressive Language-Voxel Fusion, the authors achieve state-of-the-art localization performance, demonstrating significant improvements over previous methods in handling complex relational targets.
Achieving state-of-the-art localization in 3D visual grounding by strictly aligning text with temporally ordered observations reveals the critical role of precise memory management and spatial-linguistic reasoning.
Image-based 3D visual grounding is critical for embodied agents, yet existing benchmarks suffer from loose text-observation alignment and neglect temporal ordering. We introduce SeqAlign3DVG, a novel benchmark dedicated to temporally ordered and strictly observation-aligned image-based 3D visual grounding. Unlike prior works using order-agnostic views or global point clouds, SeqAlign3DVG ensures all expressions are human-verified and strictly grounded in the provided RGB observations (single frames or ordered observation sequences). It comprises 9,622 single-view and 14,493 sequence samples featuring rich descriptions, complex relations, and multi-instance ambiguities. To tackle this benchmark, we propose a unified voxel-based pipeline featuring Relevance-Ordered Voxel Memory (ROVM) and Progressive Language-Voxel Fusion (PLVF). ROVM dynamically ranks and aggregates multi-view evidence via a conservative memory to mitigate noisy observations, while PLVF performs coarse-to-fine spatial-linguistic reasoning for precise disambiguation. Our approach achieves state-of-the-art performance under the depth-free protocol, significantly improving localization for targets defined by complex relations and appearance cues.
