Search papers, labs, and topics across Lattice.
This paper introduces Event-VLA, a novel framework that enhances Vision-Language-Action (VLA) models by integrating event streams to improve robustness in robotic manipulation under varying illumination conditions. By formulating the manipulation task as a robustness challenge for RGB-centric policies, the authors employ an action-query routing pathway to selectively aggregate event tokens, ensuring that task-relevant semantics are preserved while leveraging motion-sensitive observations. Experimental results demonstrate that Event-VLA not only maintains high manipulation performance in well-lit environments but also significantly enhances success rates in low-light and near-dark scenarios.
Event-VLA achieves robust robotic manipulation even in near-dark conditions by effectively integrating motion-sensitive event streams with traditional RGB inputs.
Vision-Language-Action (VLA) models have become an important paradigm of embodied AI. However, existing VLA models typically assume well-lit and stable indoor settings, while real-world embodied manipulation may involve degraded RGB observations caused by illumination shifts, posing critical challenges for robust robotic manipulation. To address this gap, we propose \textbf{Event-VLA}, an event-enhanced VLA framework for generalizable manipulation across varying illumination conditions. We formulate VLA-based manipulation under degraded visibility as a practical robustness problem for RGB-centric policies, and introduce event streams as an illumination-robust, motion-sensitive complementary observation to improve robustness across visibility levels. Specifically, unlike conventional multimodal fusion that directly merges event features into the global semantic token space, Event-VLA injects event information through an action-query routing pathway. It uses learnable action queries to extract task-relevant semantics from the VLA reasoning process, and selectively aggregates event tokens via gated cross-attention to construct event-aware action representations. This design preserves the pretrained RGB-language semantic priors while effectively leveraging event information for robust action prediction. Experiments in simulation and real-world deployment show that Event-VLA maintains strong manipulation performance under normal lighting and improves success rates under low-light degradation and near-dark real-world settings.
