Search papers, labs, and topics across Lattice.
This paper introduces ThinkAfford, a novel framework for task-driven 3D affordance grounding that addresses the limitations of existing methods by separating affordance proposal generation from instruction-based reasoning. By employing learnable affordance prompts and multi-level visual features, ThinkAfford generates high-recall interaction-conditioned heatmaps, allowing for fine-grained localization in cluttered scenes without relying on specific object or part names. The framework significantly outperforms existing 3D open-vocabulary and vision-language model baselines, achieving 10.69% AP50 and 25.46% AP25 on the SceneFun3D validation split, while also demonstrating improved recall and selection accuracy through its innovative Group Relative Policy Optimization approach.
ThinkAfford achieves a remarkable 10.69% AP50 by decoupling affordance proposal generation from instruction grounding, setting a new standard for 3D grounding in cluttered environments.
Task-driven 3D affordance grounding aims to localize the functional region in a cluttered 3D scene that enables an action specified by a natural-language instruction. Existing methods either predict 3D masks directly or construct them by selecting and fusing intermediate 2D/3D regions. However, they remain vulnerable to two intertwined failure modes: the predicted or selected regions may miss the target interaction area or have unsuitable granularity, while language grounding may confuse visually similar alternatives under relational instructions. To this end, we introduce ThinkAfford, which decouples high-recall affordance proposal generation from instruction-grounded reasoning. Specifically, the Affordance Proposal Generation module first uses learnable affordance prompts and multi-level visual features to predict interaction-conditioned heatmaps, extracting a variable number of fine-grained proposals without parsed object or part names as segmentation prompts. Visual-Prompted Affordance Reasoning then reasons over labeled proposal overlays using the full instruction, returning identifiers in a structured "think-then-answer" response. Moreover, Group Relative Policy Optimization uses proposal-level rewards from lifted 3D overlap to align VPAR selection with final 3D grounding. On the SceneFun3D validation split, ThinkAfford achieves 10.69% AP50 and 25.46% AP25 under the official evaluator, outperforming comparable 3D open-vocabulary and vision-language-model-based 2D-to-3D baselines. Module-level diagnostics further show that APG attains 77.5% recall at 25% intersection-over-union, while GRPO-trained VPAR achieves 72.1% selection accuracy on APG-covered queries, compared with 63.4% under supervised fine-tuning.
