Search papers, labs, and topics across Lattice.
The paper introduces Multi-Agent Vision-Language-Critic Models (MA-VLCM), a framework that leverages pre-trained vision-language models (VLMs) as centralized critics in MARL to improve sample efficiency and generalization. MA-VLCM fine-tunes a VLM, conditioned on natural language task descriptions, visual trajectory observations, and multi-agent state information, to estimate the value function of multi-agent policies. Experiments demonstrate that MA-VLCM achieves good zero-shot return estimation with various VLM backbones in both in-distribution and out-of-distribution multi-agent scenarios, while also producing compact policies suitable for resource-constrained robots.
Ditch inefficient MARL critic learning: fine-tune a pre-trained vision-language model to evaluate multi-agent behavior and drastically improve sample efficiency.
Multi-agent reinforcement learning (MARL) commonly relies on a centralized critic to estimate the value function. However, learning such a critic from scratch is highly sample-inefficient and often lacks generalization across environments. At the same time, large vision-language-action models (VLAs) trained on internet-scale data exhibit strong multimodal reasoning and zero-shot generalization capabilities, yet directly deploying them for robotic execution remains computationally prohibitive, particularly in heterogeneous multi-robot systems with diverse embodiments and resource constraints. To address these challenges, we propose Multi-Agent Vision-Language-Critic Models (MA-VLCM), a framework that replaces the learned centralized critic in MARL with a pretrained vision-language model fine-tuned to evaluate multi-agent behavior. MA-VLCM acts as a centralized critic conditioned on natural language task descriptions, visual trajectory observations, and structured multi-agent state information. By eliminating critic learning during policy optimization, our approach significantly improves sample efficiency while producing compact execution policies suitable for deployment on resource-constrained robots. Results show good zero-shot return estimation on models with differing VLM backbones on in-distribution and out-of-distribution scenarios in multi-agent team settings
