Track new results across pure and applied mathematics.
Search papers, labs, and topics across Lattice.
This paper introduces MineAmongUs, a 3D multimodal environment designed to test the deception capabilities of visual-language model (VLM) agents through both verbal and non-verbal interactions in social-deduction games. By employing ARIA, a configurable harness that allows for detailed cognitive-component ablation, the authors demonstrate that VLM agents effectively utilize joint deception strategies, with non-verbal cues proving to be more critical for success than verbal communication. The findings underscore the importance of integrating multimodal channels in evaluating agent behavior, paving the way for enhanced alignment strategies in embodied AI systems.
VLM agents excel at deception, with non-verbal cues playing a surprisingly pivotal role in their success.
Strategic deception by LLM and VLM agents has emerged as a central AI alignment and safety concern. Social-deduction games (where each player holds a hidden role and communicates with others to deduce identities) serve as the canonical testbed, particularly in multi-agent settings. Existing testbeds, however, are text-only and run on a single fixed agent configuration, missing the non-verbal sensorimotor channels treated as core by deception taxonomies and leaving it ambiguous whether an observed behavior reflects the underlying model or the surrounding harness. We introduce MineAmongUs, a 3D multimodal Among Us sandbox where imposter agents must deceive crewmates through joint verbal and non-verbal action. We also propose ARIA, a configurable VLM-agent harness that exposes five cognitive-component ablation axes; and an atom- and arc-level annotation scheme grounded in deception taxonomies and operationalized at scale by an LLM-as-a-Judge reaching near-human atom-labeling agreement. Empirical results show that VLM agents pursue imposter wins through joint verbal and non-verbal deception, with non-verbal channels emerging as the more decisive winning contributors across both harness ablation and cross-VLM evaluation. Taken together, our work opens a new path for embodied VLM-agent alignment research.