Search papers, labs, and topics across Lattice.
This paper introduces GRAFT, a novel framework that enhances online adaptation of pretrained vision-language-action (VLA) policies for fine-grained robot manipulation tasks in biomedical contexts. By leveraging region-level supervision to establish view-specific visual anchors, GRAFT effectively directs attention to crucial visual cues, thereby facilitating more efficient learning from limited real-robot interactions. The approach demonstrates a significant improvement in task success rates, achieving a 25 percentage point increase while simultaneously reducing the computational costs associated with online policy updates.
GRAFT boosts robot manipulation success rates by 25 percentage points while slashing the computational costs of online learning.
Pretrained vision-language-action (VLA) policies provide strong priors for robot manipulation, yet adapting them online to fine-grained biomedical tasks remains challenging. Task success often hinges on subtle, view-dependent visual cues, while task-level rewards provide little guidance about which regions matter, making it difficult to learn task-relevant visual grounding from limited real-robot interaction. Online adaptation is further constrained by the computational cost of VLA inference and replay-based updates. We introduce GRAFT (Grounded Reinforcement Adaptation for Fast Task Learning), a framework for efficient online VLA adaptation through grounded perception. GRAFT uses region-level supervision to learn view-specific visual anchors that focus perception on task-relevant local cues without requiring region proposals at deployment. It further combines single-step action generation with cached visual-language prefix reuse to accelerate online learning. Across four biomedical manipulation tasks, GRAFT improves success rates by 25 percentage points under matched adaptation budgets, while reducing the computational overhead of online policy updates.
