Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
We address a failure mode in offline Multi-Agent Reinforcement Learning (MARL) where generative teacher policies, trained with standard flow-based methods, produce samples interpolating between distinct coordination modes, leading to error propagation to student policies. Our proposed Mode-Support Semi-Discrete Optimal Transport (MoSDOT) method rectifies this by summarizing multimodal replay into a finite mode support and employing conditional semi-discrete optimal transport to assign noise samples to specific modes during teacher training. MoSDOT significantly enhances endpoint quality and routing consistency, especially in scenarios with complex multimodal joint behaviors.
Generative MARL teachers fail by interpolating coordination modes; a novel optimal transport approach forces them to commit to distinct strategies, boosting student performance.
Offline MARL increasingly relies on generative policies to model multimodal joint behavior, typically by distilling a centralized teacher into decentralized one-step actors under the CTDE. We identify a failure mode at the teacher training stage: standard flow-based teachers pair noise with replay targets independently, so nearby noise samples can be routed toward conflicting coordination modes. The teacher then produces samples between valid modes, and because the distillation loss regresses each local actor onto the conditional mean of the teacher's output given local input, this error is not absorbed but propagated to the student. To remove this teacher-side artifact, we propose Mode-Support Semi-Discrete Optimal Transport (MoSDOT), which summarizes multimodal replay into a finite mode support with prescribed capacities and uses conditional semi-discrete optimal transport to assign each noise sample to a single mode before teacher training. We additionally study a shared-randomness variant that uses a shared noise component at execution to expose the residual gap intrinsic to strict-product execution. On controlled diagnostics and offline MARL benchmarks, MoSDOT improves endpoint quality and routing consistency, particularly on datasets exhibiting multimodal joint behavior.