Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper investigates the training of a single diffusion policy from scratch using reinforcement learning for multi-task block-pushing tasks involving various shapes. By employing a novel policy loss function based on a reweighted evidence lower bound and integrating reverse curriculum generation, the authors successfully address exploration challenges in a sparse-reward environment. The results demonstrate that the trained diffusion policy can achieve zero-shot transfer to real-world scenarios, effectively adapting to diverse conditions such as goal positions and block characteristics.
Achieving zero-shot transfer of a single diffusion policy for multi-task block pushing could revolutionize robotic manipulation across varied real-world environments.
Diffusion policies have shown promising empirical performance in representing and learning complex maneuvers for robots using behavior cloning (BC). In this paper, we explore training diffusion policies from scratch using reinforcement learning (RL) for multi-task robotic manipulation. Specifically, we aim to train a single diffusion policy for block-pushing tasks with multiple shapes. The proposed framework features a simple policy loss function, which is a reweighted evidence lower bound used in BC-based diffusion policy training and can seamlessly serve as the policy learning module in RL algorithms. To address the exploration challenges arising from the absence of demonstrations, we incorporate reverse curriculum generation and objective-centric representations. Combined with the expressiveness of diffusion policies, our design supports learning of multi-task block-pushing policies in our sparse-reward simulation setting. We further evaluate whether the trained diffusion policy transfers in zero-shot to real-world tasks under varying environmental conditions including goal positions, block shapes, block weights and surface friction, providing evidence that this pipeline can transfer to our real-world block-pushing setup under the tested variations.