Search papers, labs, and topics across Lattice.
The paper introduces Localized Dynamics-Aware Domain Adaptation (LoDADA) for off-dynamics offline RL, addressing the challenge of learning a target policy from limited target data and abundant source data with different transition dynamics. LoDADA clusters transitions from both datasets and estimates dynamics discrepancy at the cluster level using domain discrimination, enabling fine-grained source data selection based on localized dynamics mismatch. Empirical results demonstrate that LoDADA outperforms existing methods by effectively leveraging localized distribution similarities and dissimilarities across domains.
By clustering transitions and estimating cluster-level dynamics discrepancy, LoDADA offers a fine-grained and scalable data selection strategy that outperforms existing off-dynamics offline RL methods.
Off-dynamics offline reinforcement learning (RL) aims to learn a policy for a target domain using limited target data and abundant source data collected under different transition dynamics. Existing methods typically address dynamics mismatch either globally over the state space or via pointwise data filtering; these approaches can miss localized cross-domain similarities or incur high computational cost. We propose Localized Dynamics-Aware Domain Adaptation (LoDADA), which exploits localized dynamics mismatch to better reuse source data. LoDADA clusters transitions from source and target datasets and estimates cluster-level dynamics discrepancy via domain discrimination. Source transitions from clusters with small discrepancy are retained, while those from clusters with large discrepancy are filtered out. This yields a fine-grained and scalable data selection strategy that avoids overly coarse global assumptions and expensive per-sample filtering. We provide theoretical insights and extensive experiments across environments with diverse global and local dynamics shifts. Results show that LoDADA consistently outperforms state-of-the-art off-dynamics offline RL methods by better leveraging localized distribution mismatch.
