Search papers, labs, and topics across Lattice.
This paper addresses the challenge of shortcut learning in robotic foundation models, where policies exploit non-causal correlations instead of relevant visual structures. The authors introduce Artificial Foveated Perception (AFP), a lightweight module that generates task-conditioned masks to guide policy attention towards critical visual regions during fine-tuning. The results demonstrate that AFP significantly reduces fine-tuning time, mitigates overfitting, and enhances generalization in varying environments, suggesting a promising approach for improving the robustness and efficiency of robotic models.
Task-conditioned foveated perception can drastically enhance the robustness and efficiency of robotic foundation models by aligning policy learning with relevant visual evidence.
Robotic foundation models have recently made substantial progress in multi-task capability, cross-embodiment transfer, and language-conditioned control. Yet robust deployment across diverse real-world settings remains difficult, in part because policies often fail to distinguish causally relevant visual structure from spurious scene-level correlations. We identify this failure mode as shortcut learning: the tendency to exploit predictive but non-causal correlations in the training distribution rather than the task-relevant visual evidence that determines successful action. Although shortcut learning has been extensively studied in computer vision and broader machine learning, its role in robotic foundation models remains comparatively underexplored. We propose Artificial Foveated Perception (AFP), a lightweight, policy-agnostic module that takes the same vision and language inputs as Vision-Language-Action and World Action Model pipelines and predicts task-conditioned masks over relevant objects, the robot, and other action-critical regions. We use these masks primarily as an auxiliary grounding signal during fine-tuning, aligning policy attention with task-relevant regions while leaving the core architecture unchanged. After fine-tuning, the policy executes on the original observation stream without requiring AFP in the control loop. We evaluate AFP across state-of-the-art robotic foundation models and show that foveated perception reduces fine-tuning time, suppresses overfitting, and improves generalization under environmental perturbations. Ablations over mask quality and grounding-loss design further show that these gains arise from directing policy learning toward task-relevant visual evidence. These results suggest that task-conditioned foveated perception is a practical mechanism for making robotic foundation models more robust, data-efficient, and scalable.
