Track manipulation, locomotion and embodied systems.
Search papers, labs, and topics across Lattice.
This paper investigates how to leverage demonstration quality in Vision-Language-Action (VLA) pretraining by comparing weighting, filtering, and conditioning strategies across various reward definitions and label granularities using a benchmark of suboptimal human demonstrations and 24 target tasks. The study finds that approach-aware episode conditioning enhances performance over behavioral cloning, with conditioning and filtering showing differential benefits based on label granularity and model architecture. The introduction of AXIS-Bench, a benchmark with replayable simulation trajectories, enables controlled comparisons and demonstrates that simulation pretraining significantly boosts real-world task success.
Quality-aware pretraining strategies, particularly approach-aware episode conditioning, can significantly outperform plain behavioral cloning in VLA tasks, with optimal choices depending on label granularity and model architecture.
How should demonstration quality be used during vision-language-action (VLA) pretraining to improve performance after target-task fine-tuning? We compare weighting, filtering, and conditioning across reward definitions and label granularities using 5,000 naturally suboptimal human demonstrations and 24 target tasks. Within each VLA, we match demonstrations and training budgets. Additional comparisons examine pretraining data size and the roles of quality tags during fine-tuning and inference. Across three VLAs, we find that approach-aware episode conditioning improves mean success over plain behavioral cloning; conditioning and filtering favor episode and chunk labels, respectively, on average, with model-dependent reversals; and constant-tag fine-tuning improves mean success after conditioned pretraining with either tagged or untagged inference. We introduce AXIS-Bench to support these controlled comparisons. Built on crowdsourced simulation demonstrations, it provides recorded object and contact states for recomputing quality labels on shared trajectories. Replayable trajectories support workspace-aligned rendering and user-defined real-world target tasks. In our real-world experiment, simulation pretraining raises success from 9.6\% to 31.7\%. Project page: https://axis-bench.github.io/.
CMU ML
NUS