Search papers, labs, and topics across Lattice.
ATHENA introduces a novel influence function framework designed for efficient data curation in billion-parameter Vision-Language-Action (VLA) models, addressing the computational challenges of multitask learning. By leveraging the Kronecker structure of linear-layer gradients and employing a rank-r Random Truncated Approximation for Hessian inversion, ATHENA achieves a remarkable 313.4x speedup in influence computation. Evaluations on RoboTwin 2.0 and real-robot tasks reveal that ATHENA can match or surpass the performance of full-data joint fine-tuning while utilizing only 50% of the demonstrations in simulation and 66.7% in real-world applications.
ATHENA accelerates influence computation for billion-parameter models, achieving a staggering 313.4x speedup while maintaining or improving task performance with significantly less data.
In robot imitation learning, influence functions provide a principled approach to quantify each demonstration's effect on robot task outcomes, yet scaling them to billion-parameter Vision-Language-Action (VLA) models is limited by computational and multitask bottlenecks. To this end, we propose ATHENA, an influence function framework tailored for multitask VLA data curation at a billion-parameter scale. Concretely, it leverages the Kronecker structure of linear-layer gradients to reduce projection cost, and approximates dense Hessian inversion with a rank-r Random Truncated Approximation, achieving about a 313.4x speedup in influence computation. Furthermore, ATHENA formulates global and local interactive influence to balance data curation across 50 jointly trained tasks. Extensive evaluations on RoboTwin 2.0 and real-robot deployment, covering 9.34 and 6.90 hours of demonstrations, respectively, show that ATHENA matches or exceeds full-data joint fine-tuning using only 50% of demonstrations in simulation and 66.7% of data across six real-robot tasks. Overall, ATHENA demonstrates its effectiveness for data curation in billion-parameter multitask VLA fine-tuning.
Tsinghua AI