Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper establishes finite-sample fast convergence rates for adversarial imitation learning (AIL) under general function approximation by jointly regularizing both the discriminator reward and the generator policy. While empirical methods like GAIL rely heavily on dual regularization, formal theory has lacked guarantees that achieve optimal sample efficiency simultaneously for expert demonstrations and online environment steps. Their model-free algorithm, Dually Regularized AIL, achieves a fast $\widetilde{O}(1/K + 1/N)$ bound on the regularized imitation gap, establishing the first simultaneous $\widetilde{O}(1/\varepsilon)$ sample complexity in both online interactions ($K$) and expert trajectories ($N$), even with stochastic experts.
Adversarial imitation learning can achieve fast $\widetilde{O}(1/\varepsilon)$ sample complexity across both expert demonstrations and online environment interactions—provided you jointly regularize the policy and the reward.
We study adversarial imitation learning (AIL), in which an agent learns to imitate expert demonstrations by optimizing a policy against an adversarial reward that distinguishes expert and learner behavior. Historically, reward regularization and entropy-based policy regularization are key components of empirically successful methods such as GAIL and LS-IQ, yet their finite-sample benefits remain underexplored. We establish fast rates for jointly regularized AIL in finite-horizon Markov decision processes with general function approximation. Our model-free algorithm, Dually Regularized AIL, combines KL policy regularization with a quadratic reward penalty weighted by expert and learner occupancies. With K online episodes and N expert trajectories, we prove a $\widetilde{O}\left(\frac{1}{K}+\frac{1}{N}\right)$ bound on the regularized imitation gap for fixed regularization parameters. Our analysis combines an online mirror descent construction for general convex reward classes to control estimation error from finite expert data and stochastic learner feedback, with a sharp analysis of optimistic KL-regularized policy learning. To the best of our knowledge, Dually Regularized AIL is the first algorithm to simultaneously achieve $\widetilde{O}\left(\frac{1}ε\right)$ sample complexity in both expert demonstrations and online interactions for this regularized AIL objective, even with stochastic experts. These results provide a rigorous characterization of the complementary statistical benefits of reward and policy regularization in AIL.