Search papers, labs, and topics across Lattice.
This paper introduces CoDrift, a novel compositional framework for offline reinforcement learning that integrates multiple objective-level fields to enhance policy learning. By treating each objective as a motion field, CoDrift effectively combines state-dependent behavioral structures, action pooling, and value-driven action selection into a unified policy field. The framework demonstrates superior performance across 73 tasks in both offline and offline-to-online settings, achieving the best average rank compared to existing state-of-the-art methods.
CoDrift outperforms leading offline RL methods by harmonizing multiple learning objectives into a single, efficient policy generator.
Offline reinforcement learning is intrinsically multi-objective: a policy must remain compatible with the behavioral support of a fixed dataset while preferentially selecting high-value actions. We recast these objectives in a common form by viewing each as an action-space motion field that specifies how generated actions should move. This perspective enables heterogeneous learning objectives to be combined directly through field composition. Inspired by drifting models, we propose CoDrift, a compositional framework for one-step generative policy learning. CoDrift combines three objective-level fields into a unified policy field. The conditional field preserves state-dependent behavioral structure, while the marginal field pools actions across states to provide a more stable generative signal in the single-positive-sample regime of continuous-control offline RL. The value field moves generated actions toward higher-value regions. The composed field is absorbed into a stochastic generator that produces an action with a single forward pass at deployment. We evaluate CoDrift on 73 tasks from OGBench and D4RL in both offline and offline-to-online settings. CoDrift compares favorably with state-of-the-art methods and achieves the best average rank in both settings.
