Search papers, labs, and topics across Lattice.
This paper introduces the Aligned Refinement Policy (ARP), a novel framework that enhances discrete skill-based imitation learning for robotic manipulation by integrating visual-semantic grounding with execution-level refinement. By employing a visual-action alignment objective and a lightweight Iterative Residual Head, ARP effectively addresses the limitations of existing methods, which often struggle with precision and weak visual grounding. Experimental results demonstrate that ARP achieves state-of-the-art performance on benchmark tasks and significantly outperforms baseline methods in real-robot scenarios, showcasing its practical applicability in complex manipulation tasks.
ARP not only aligns visual observations with action representations but also refines execution precision, leading to unprecedented performance in robotic manipulation tasks.
Learning visuomotor policies for long-horizon manipulation remains a fundamental challenge. Recent skill-based imitation learning methods based on discrete quantization have shown promising results by representing complex behaviors as temporally extended skills. However, most existing approaches primarily encode action trajectories into latent skills, yielding weak visual-semantic grounding and limiting the ability to leverage visual observations for skill selection. Moreover, discrete tokenization inevitably incurs precision errors during continuous action generation. To alleviate these issues, we propose Aligned Refinement Policy (ARP), a discrete-skill framework that couples semantic grounding with execution-level refinement. Specifically, ARP introduces (i) a visual--action alignment objective that contrastively aligns visual embeddings with pre-quantized action representations in a shared latent space while preserving a state-independent skill decoder, and (ii) a lightweight Iterative Residual Head (IRH) that performs a two-step refinement to recover fine-grained control for precise execution. Extensive experiments show that ARP achieves state-of-the-art performance on the LIBERO and Meta-World benchmarks. Moreover, real-robot experiments on the Kuavo 4 Pro humanoid platform further validate its effectiveness, yielding consistent performance gains over several baselines on two challenging manipulation tasks.
Tsinghua AI