Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper addresses distribution shift in imitation learning by proposing a two-phase framework: a robust offline learning phase using supplementary demonstrations guided by a discriminator, followed by an adaptive online inference phase with self-supervised imitation learning. The discriminator loss effectively expands the state-action coverage during offline training, improving robustness to unseen states. Experiments in MuJoCo environments show the proposed method outperforms baselines in both robustness to distribution shift and adaptation to new environments.
Mitigating distribution shift in robotics control is now more achievable thanks to a novel offline-to-online imitation learning framework that leverages a discriminator to expand state-action coverage and self-supervised learning for online adaptation.
Distribution shift in imitation learning refers to the problem that the agent cannot plan proper actions for a state that has not been visited during the training. This problem can be largely attributed to the inherently narrow state-action coverage provided by expert demonstrations over the full environment. In this paper, we propose a robust offline to adaptive online imitation learning framework that handles the distribution shift problem in a lifelong, multi-phase scheme. In the offline learning phase, we leverage supplementary demonstrations to broaden the state-action coverage of the policy by utilizing a discriminator to effectively train the policy with supplementary demonstrations, thereby enhancing the robustness of the policy to distribution shift. In the subsequent online inference phase, our framework detects the occurrence of distribution shift and conducts self-supervised imitation learning from online experiences to adapt the policy to the online environments. Through extensive evaluations in MuJoCo environments, we demonstrate that our method exhibits better robustness to distribution shift and better adaptation performance to online environments than the baseline algorithms, which indicates superior performance of our framework against the distribution shift.