Search papers, labs, and topics across Lattice.
LangMARL addresses the multi-agent credit assignment problem in LLM-based multi-agent reinforcement learning by introducing agent-level language credit assignment and policy gradient evolution in language space. The framework also summarizes task-relevant causal relations from replayed trajectories to provide dense feedback. Experiments across diverse cooperative multi-agent tasks demonstrate improved sample efficiency, interpretability, and generalization compared to existing LLM agents.
LLM agents can learn to cooperate far more efficiently by borrowing credit assignment techniques from classic multi-agent RL.
Large language model (LLM) agents struggle to autonomously evolve coordination strategies in dynamic environments, largely because coarse global outcomes obscure the causal signals needed for local policy refinement. We identify this bottleneck as a multi-agent credit assignment problem, which has long been studied in classical multi-agent reinforcement learning (MARL) but remains underaddressed in LLM-based systems. Building on this observation, we propose LangMARL, a framework that brings credit assignment and policy gradient evolution from cooperative MARL into the language space. LangMARL introduces agent-level language credit assignment, pioneers gradient evolution in language space for policy improvement, and summarizes task-relevant causal relations from replayed trajectories to provide dense feedback and improve convergence under sparse rewards. Extensive experiments across diverse cooperative multi-agent tasks demonstrate improved sample efficiency, interpretability, and strong generalization.
