Search papers, labs, and topics across Lattice.
This paper introduces LIBERO-VIFO, a benchmark designed to evaluate the capability and safety of visual cue following in Vision-Language-Action (VLA) models. The study reveals that while current VLA models can execute tasks indicated by visual cues without accompanying language instructions, they also exhibit a concerning tendency to follow unauthorized cues, highlighting significant safety risks. By systematically assessing both authorized and unauthorized cue following across diverse visual cue families, the research establishes a new framework for understanding visual-centric safety in VLA systems.
Current VLA models can execute tasks based solely on visual cues, but they also risk following unauthorized cues, raising critical safety concerns.
Visual cues are increasingly adopted to guide robot learning, but whether Vision-Language-Action (VLA) models can reliably follow authorized cues while disregarding unauthorized ones remains unclear. Existing work covers only a narrow range of cue forms and focuses on final task success, providing only a coarse assessment of cue-following capability. Treating all visual cues as authorized also leaves safety risks of unauthorized following unexplored. To address these gaps, we introduce LIBERO-VIFO, a benchmark to evaluate both the capability and safety of visual cue following in VLA models. LIBERO-VIFO defines eight visual cue families spanning diverse forms. A total of four protocols in two parts are defined: Part I tests cue understanding and authorized following, while Part II evaluates unauthorized visual cue following under language-cue conflict and empty language conditions. Evaluating seven VLA models reveals that although visual cue understanding does not reliably translate into execution, current VLAs are able to execute cue-indicated tasks without language instruction, exposing an emerging risk of unauthorized visual cue following. Extended experiments on scene-instantiated cues, safety-critical settings, and real-robot deployment corroborate these findings. LIBERO-VIFO brings both the capability and safety of visual cue following into systematic evaluation, establishing visual-centric safety as a new perspective for the VLA community.
