Search papers, labs, and topics across Lattice.
This paper investigates the issue of stagnant neurons in Multi-Agent Reinforcement Learning (MARL) value factorization methods, which lose their ability to adapt to new tasks due to negligible gradient updates. The authors introduce a novel approach called Knowledge-retentive Neuron-level PlastIcity Focusing InjEction (KNIFE), which replaces stagnant neurons with a composite unit designed to retain knowledge while restoring learning capacity. Experimental results across various game scenarios show that KNIFE significantly outperforms existing plasticity injection methods, highlighting its effectiveness in enhancing adaptability in MARL systems.
Stagnant neurons in MARL can cripple learning, but KNIFE revitalizes them, leading to substantial performance gains in complex environments.
Multi-Agent Reinforcement Learning (MARL) value factorization methods can suffer from a loss of plasticity, gradually failing to adapt when transferring to new task instances. We trace this issue to stagnant neurons, units whose gradient updates become negligibly small relative to their weights, thereby hindering learning. While existing plasticity injection methods exist, they prove ineffective for such neurons. To address this, we propose Knowledge-retentive Neuron-level PlastIcity Focusing InjEction (KNIFE), a novel method that directly targets stagnant neurons. KNIFE replaces each stagnant neuron with a composite unit comprising three specialized components: a frozen knowledge neuron to preserve acquired knowledge, a re-initialized active neuron to restore learning capacity, and a compensation neuron to ensure the combined output matches the original, thus maintaining previous learned cooperation knowledge. Extensive experiments on SMACv2, predator-prey, and matrix games demonstrate that KNIFE significantly outperforms state-of-the-art plasticity injection methods.
