Search papers, labs, and topics across Lattice.
This paper introduces DF-ExpEnse, an exploration technique that enhances the sample efficiency of fine-tuning pretrained generative control policies by improving the quality of online experience collection. By leveraging multimodal modeling and an ensemble of critics, DF-ExpEnse effectively balances exploration and action quality, facilitating better decision-making in robotic tasks. Experimental results demonstrate that DF-ExpEnse consistently outperforms traditional finetuning methods across various manipulation and locomotion tasks, showcasing its potential for real-world applications in robotics.
DF-ExpEnse boosts sample efficiency in robotic fine-tuning by intelligently balancing exploration and quality, outperforming traditional methods across diverse tasks.
A natural recipe for intelligent robotic decision-making is initializing from pretrained generative control policies, which have summarized offline experience, and adapting them to self-collected online experience. We present DF-ExpEnse, an exploration technique that improves the quality of online experience collection, thus increasing finetuning sample-efficiency. DF-ExpEnse leverages the multimodal modeling capabilities of the generative control policy to create an expressive and tractably evaluatable candidate set. It then utilizes an ensemble of critics to identify the action that best balances quality with high exploration interest. In fleet settings, DF-ExpEnse further enables cross-agent communication to facilitate collaborative exploration as a group. DF-ExpEnse can be seamlessly integrated with existing strategies that finetune pretrained generative control policies via reinforcement learning. We experimentally validate consistent sample-efficiency benefits through DF-ExpEnse across a variety of manipulation and locomotion tasks, compared to default finetuning and alternative action selection schemes. Project can be found at https://df-expense.github.io.
