Search papers, labs, and topics across Lattice.
This paper investigates the impact of scene-induced occlusion on Vision-Language-Action (VLA) models, revealing significant performance drops in standard benchmarks when objects are not fully visible. To combat this challenge, the authors introduce LIBERO-Occ, an extension of the LIBERO framework, and propose a novel method called Viewpoint Imagination (VIM), which generates additional views from occluded observations to enhance action prediction. Experimental results demonstrate that VIM significantly improves the robustness of VLA models across various tasks and occlusion scenarios without the need for extra cameras during deployment.
VLA models can lose up to 40% of their effectiveness in real-world scenarios due to occlusion, but viewpoint imagination offers a powerful solution to this pervasive challenge.
Vision-Language-Action (VLA) models achieve strong performance on standard manipulation benchmarks, but most evaluations assume that task-relevant objects are fully visible. This assumption often fails in realistic settings, where occlusion makes manipulation partially observable. In this paper, we study \textit{scene-induced occlusion} as a fundamental challenge for VLA models and introduce \textbf{LIBERO-Occ}, an occlusion-oriented extension of LIBERO. Experiments show that state-of-the-art VLAs suffer substantial performance degradation under occlusion. To address this issue, we propose \textbf{Viewpoint Imagination (VIM)}, which generates a complementary view from an occluded primary observation and conditions action prediction on both observed and imagined evidence. VIM improves robustness across task suites, occlusion types, and severity levels without requiring additional cameras at deployment time, suggesting that viewpoint imagination is an promising mechanism for perception completion in partially observable manipulation. Our benchmark and corresponding code are available at: \href{https://github.com/litsh/Libero-Occ}{https://github.com/litsh/Libero-Occ}.
