Search papers, labs, and topics across Lattice.
This paper introduces APEIRIA, a neuro-symbolic 3D multi-modal language model that integrates symbolic reasoning with the flexibility of end-to-end models. By employing a three-stage curriculum that includes 3D perception alignment, chain-of-thought supervised fine-tuning, and reinforcement learning, APEIRIA effectively distills reasoning patterns from neuro-symbolic programs into a format suitable for open-set concepts. The results demonstrate that APEIRIA not only outperforms existing neuro-symbolic 3D methods but also matches the performance of state-of-the-art 3D multi-modal language models in spatial reasoning tasks.
APEIRIA bridges the gap between interpretable neuro-symbolic reasoning and the flexibility of multi-modal language models, achieving superior performance in 3D spatial reasoning.
Current 3D spatial reasoning methods face a fundamental trade-off: neuro-symbolic 3D (NS3D) concept learners achieve interpretable reasoning through compositional programs but are constrained to closed-set concept vocabularies and simple programs; end-to-end 3D multi-modal LLMs (3D MLLMs) could handle complex natural language and open-vocabulary concepts but suffer from black-box reasoning without explicit spatial verification. We introduce APEIRIA, a neuro-symbolic 3D MLLM to bridge two paradigms by distilling symbolic reasoning patterns into MLLMs with natural language chain-of-thought. Our three-stage curriculum progressively builds reasoning capabilities: a) 3D perception alignment grounds object visual-geometric features to the LLM, b) CoT-SFT teaches query decomposition and stepwise verification from symbolic program traces, and c) CoT-RL extends reasoning patterns to open-set concepts and deeply nested instructions. By transferring reasoning patterns rather than concept-specific knowledge, APEIRIA preserves key NS3D virtues: transparent reasoning and modular interchangeability of planning and perception components. Evaluations on grounding, question answering, and captioning show that APEIRIA surpasses prior NS3D methods and matches state-of-the-art 3D MLLMs on 3D spatial reasoning datasets, unifying symbolic methods' systematic reasoning with MLLMs' flexibility. Code is available at https://github.com/oceanflowlab/APEIRIA.
Tsinghua AI