Search papers, labs, and topics across Lattice.
This paper introduces ViewMind3D, a modular framework for 3D question answering (3D-QA) that operates without the need for training or fine-tuning, thereby enhancing scalability and applicability in real-world scenarios. By decomposing the 3D-QA task into four interpretable components, the framework enables effective spatial reasoning using multi-view observations while avoiding the complexities of complete 3D reconstruction. Experimental results indicate that ViewMind3D achieves competitive performance on benchmark datasets, particularly excelling in spatially grounded question types, with an overall accuracy of 50.8% and a CIDEr score of 73.4 on ScanQA.
Achieving robust 3D spatial reasoning without any training, ViewMind3D redefines the landscape of 3D question answering.
Recent advances in large language models (LLMs) and vision-language models (VLMs) have enabled new possibilities for 3D question answering (3D-QA), a key capability for embodied AI and robotic perception. However, most existing methods rely on 3D-specific training or fine-tuning with costly annotations, limiting their scalability and real-world applicability. We present \textbf{ViewMind3D}, a fully training-free and modular framework for 3D spatial reasoning over multi-view observations of a scene without requiring complete 3D reconstruction. The framework decomposes the 3D-QA task into four interpretable components: (1) question-driven multi-view selection, (2) guided visual grounding with language-conditioned object cues, (3) spatial context encoding via a bird's-eye-view (BEV) viewpoint indicator, and (4) structured answer generation through role-based reasoning. This design enables structured, robust, and interpretable reasoning without requiring model tuning. Experimental results on ScanQA and SQA3D show that ViewMind3D achieves competitive performance compared to prior training-free and fine-tuned 3D-LLMs. In particular, our method improves performance on spatially grounded question types, such as ``What''questions in SQA3D, while maintaining strong overall accuracy (50.8\%) and achieving 73.4 CIDEr on ScanQA. These results demonstrate that effective 3D reasoning can be achieved through modular orchestration of general-purpose LLMs and VLMs for robotic perception in real-world environments.
