Search papers, labs, and topics across Lattice.
This paper investigates the application of weight pruning techniques to Vision-Language Models (VLMs) for enhancing low-latency inference in egocentric visual understanding, crucial for human-robot collaboration. The authors reveal that traditional pruning methods often maintain evidence localization at the expense of prediction accuracy, which poses risks in real-time interactions. They introduce a rationale-informed pruning strategy that not only improves prediction accuracy but also ensures outputs are both accurate and evidentially grounded, achieving superior performance on benchmark datasets.
Traditional pruning methods compromise prediction accuracy, but a new rationale-informed strategy ensures both accuracy and evidence alignment in VLMs.
The rapid rise of Vision-Language Models (VLMs) in egocentric visual understanding has made low-latency inference in human-robot collaborative (HRC) tasks increasingly critical. Weight pruning techniques developed for VLMs to shrink model size and computation can be readily applied to satisfy the efficiency demands of on-board processing and real-time interactive robotics. Moreover, safe human-robot interaction demands pruning strategies that preserve doubly-correct predictions; outputs must be both accurate and evidentially grounded to mitigate risks and ensure user trust. In this paper, we present a new study of VLM pruning through the lens of doubly-correct prediction. Our experiments surprisingly show that existing pruning methods often preserve the right evidence localization but undermine correct prediction. To address this, we propose a rationale-informed pruning strategy that better aligns evidence with decisions. Benchmark results on egocentric video datasets demonstrate that our method not only achieves the highest prediction accuracy but also outperforms existing approaches in attaining doubly-correct predictions. We aim to stimulate research on efficient and reliable VLMs, ensuring accuracy-driven advances align with the transparency, auditability, and safety required for responsible human-robot interaction and embodied intelligence.
