Track new results across pure and applied mathematics.
Search papers, labs, and topics across Lattice.
This paper introduces CometVLA, a novel framework that enhances vision-language-action (VLA) models by integrating an embodied physical VQA corpus and benchmark, specifically designed to align with robot action data. By employing Global Action Prior (GAP) tokens, the model effectively isolates motion regularities, allowing it to leverage physical commonsense without compromising the integrity of the pre-trained VLM backbone. The results show that CometVLA significantly outperforms existing VLA baselines in real-world manipulation tasks and simulations, confirming that improved physical understanding during pre-training translates to better action generation performance.
CometVLA reveals that integrating embodied physical commonsense into VLA models leads to substantial improvements in real-world manipulation tasks, challenging the notion that disembodied training is sufficient.
Vision-language-action (VLA) models remain brittle in manipulation tasks that require physical commonsense. Current physical VQA data is typically disembodied and misaligned with robot action domains. Egocentric videos are used only as auxiliary pre-training. It remains unclear whether improved VLM physical understanding actually benefits downstream action generation. Therefore, we present CometVLA to close this gap. We construct CometData and CometBench, an embodied physical VQA corpus and benchmark strictly aligned with the robot's action data and embodiment. We introduce Global Action Prior (GAP) tokens, a compact learnable bottleneck that isolates task-agnostic motion regularities and lets the action head consume physical commonsense without corrupting the pre-trained VLM backbone. We co-train CometVLA across the embodied data pyramid, spanning teleoperation, simulation, egocentric trajectories, and VQA layers. On real-world manipulation tasks and RoboTwin simulation, CometVLA consistently outperforms strong VLA baselines. Correlation analysis shows that stronger VLM performance on CometBench indicates higher VLA success rates. Results demonstrate that physical understanding pre-training genuinely benefits downstream manipulation.