Search papers, labs, and topics across Lattice.
This paper introduces SAFE, a novel framework for multi-agent reinforcement learning (MARL) that effectively applies counterfactual credit assignment to continuous-action cooperative tasks. By utilizing a self-evolving default action sampled from agents' experience buffers, SAFE eliminates biases associated with Monte Carlo sampling and guarantees convergence to local optima. Experimental results on cooperative vehicular tasks show that SAFE consistently outperforms existing state-of-the-art models, highlighting its effectiveness in continuous action spaces.
A self-evolving default action can revolutionize how agents collaborate in continuous-action environments, ensuring unbiased learning and optimal convergence.
Counterfactual credit assignment has proven effective in multi-agent reinforcement learning (MARL) for discrete action spaces, yet its extension to continuous-action cooperative tasks remains challenging. Existing methods that approximate the counterfactual baseline via Monte Carlo sampling often introduce bias into policy gradients and fail to guarantee convergence to local optima, as the sampled actions may not have been sufficiently trained. To address these limitations, we propose SAFE, a novel MARL framework that employs a counterfactual baseline conditioned on a self-evolving default action sampled from each agent's experience buffer. This design naturally extends to continuous action spaces without relying on additional simulations, reward models, or environment-specific prior knowledge. The baseline accurately quantifies each agent's contribution, and introduces no bias into the deterministic policy gradient, ensuring convergence to local optima. Extensive experiments on cooperative vehicular tasks demonstrate that SAFE consistently outperforms state-of-the-art models.
