Search papers, labs, and topics across Lattice.
This study investigates the mechanisms behind the performance improvements in robotic control policies achieved through action chunking, revealing that traditional explanations like temporal consistency and horizon reduction do not fully account for its efficacy. Instead, the authors demonstrate that action chunking enhances non-Markovian expressivity and reduces compounding error, with these advantages being replicable through delayed policies that leverage past observations. Additionally, they introduce the concept of implicit ensembling, showing that action chunking policies can mimic the robustness of model ensembles, leading to superior generalization in both simulated and real-world environments.
Action chunking's real power lies in its ability to create implicit ensembles, boosting robustness and generalization beyond traditional policy approaches.
Action chunking---predicting and executing multiple actions instead of a single action---has proven to be a critical component for learning effective robotic control policies. However, our precise understanding of why action chunking improves performance has remained limited. In this work we seek to close this gap. Through rigorous experimental evaluations in both simulated and real-world settings, we show that existing hypotheses for the success of action chunking---temporal consistency, horizon reduction, and representation learning---fail to explain the success of action chunking. Instead, we find that action chunking benefits from greater non-Markovian expressivity and reduced compounding error compared to Markovian policies, but, in many settings of interest, these effects can be fully captured by delayed policies, which at each step predict a single action based on the observation $k$ steps in the past. We then show that there exists an additional benefit of action chunking that we refer to as implicit ensembling. In particular, by learning a diversity of temporal relationships (that is, $a_t | o_t, a_t | o_{t-1}, \ldots$), action-chunked policies exhibit behavior matching that of a model ensemble, increasing their robustness and generalization ability over policies that only learn a single temporal relationship. Building on these insights, we show that in simulated and real-world robotic control settings, we can match the performance of action chunking without action chunking---by deploying an action chunking policy as an ensemble of policies with randomized delays. Furthermore, we propose a policy class that amplifies the benefits of action chunking by explicitly instantiating an ensemble, and which we show significantly improves over the performance of action chunking in many domains.
BAIR