Search papers, labs, and topics across Lattice.
This paper introduces VAORA, a novel reward design aimed at enhancing vision-language models' (VLMs) ability to generalize in interactive physical reasoning tasks. By implementing two complementary rewards鈥擵isual Alignment Reward and Visual-Action Alignment Reward鈥攖he approach effectively mitigates hallucinated reasoning and aligns the model's reasoning with its actions. Experimental results on PHYRE and Virtual Tool demonstrate that VAORA significantly improves performance in novel tasks and unseen environments, indicating a pathway toward more grounded and generalizable physical intelligence in VLMs.
VAORA reduces hallucinated reasoning in VLMs by aligning visual context with action outcomes, leading to better generalization in unseen tasks.
Vision-language models (VLMs) struggle to generalize in interactive physical reasoning, particularly under unseen tasks and environments. Two key failure modes are prominent: hallucinated chain-of-thought (CoT) reasoning that contradicts physical reality, and misalignment between the model's reasoning and actions. We present VAORA (Visual Action Outcome Reasoning Alignment), a novel reward design that directly addresses both issues. VAORA introduces two complementary rewards: Visual Alignment Reward, which anchors VLM reasoning to the visual context independent of the agent action itself, and Visual-Action Alignment Reward, which grounds reasoning in the visual outcome induced by the model's action. Together, these rewards suppress hallucinated CoT and reduce the gap between reasoning and behavior. To improve training stability, we further employ smooth, dense rewards by estimating success probabilities using a pre-trained in-domain expert agent. Experiments on PHYRE and Virtual Tool support our performances across novel-task and unseen-environment settings, confirming that grounded and generalizable physical intelligence can be induced through VAORA.
