Search papers, labs, and topics across Lattice.
This paper introduces HD-ExpIt, a framework for iteratively refining hierarchical diffusion policies for language-conditioned manipulation tasks using environment feedback. HD-ExpIt employs a diffusion-based planner to explore successful behaviors, which are then distilled back into the hierarchical policy, enabling the planner to learn the controller's capabilities. Experiments on the CALVIN benchmark demonstrate that HD-ExpIt significantly improves performance compared to hierarchical policies trained only on offline data, achieving state-of-the-art results among methods trained from scratch.
Stop hand-designing subgoals: HD-ExpIt uses environment feedback to iteratively refine hierarchical diffusion policies, teaching the planner to generate subgoals that the controller can actually achieve.
Hierarchical policies for language-conditioned manipulation decompose tasks into subgoals, where a high-level planner guides a low-level controller. However, these hierarchical agents often fail because the planner generates subgoals without considering the actual limitations of the controller. Existing solutions attempt to bridge this gap via intermediate modules or shared representations, but they remain limited by their reliance on fixed offline datasets. We propose HD-ExpIt, a framework for iterative fine-tuning of hierarchical diffusion policies via environment feedback. HD-ExpIt organizes training into a self-reinforcing cycle: it utilizes diffusion-based planning to autonomously discover successful behaviors, which are then distilled back into the hierarchical policy. This loop enables both components to improve while implicitly grounding the planner in the controller's actual capabilities without requiring explicit proxy models. Empirically, HD-ExpIt significantly improves hierarchical policies trained solely on offline data, achieving state-of-the-art performance on the long-horizon CALVIN benchmark among methods trained from scratch.
