Search papers, labs, and topics across Lattice.
This paper investigates the effects of explicit personality conditioning in Multimodal Large Language Models (MLLMs) through a systematic evaluation framework that includes single-personality induction, multi-personality induction, and personality switching. The findings reveal that while personality induction enhances image captioning performance, it can detrimentally affect tasks requiring precise reasoning, such as visual question answering (VQA). Additionally, the study uncovers that model behavior is influenced by both prior and current personality constraints, highlighting the intricate dynamics of personality modeling in MLLMs.
Personality induction boosts image captioning but can hinder reasoning tasks, revealing a complex interplay in MLLM behavior that demands tailored approaches.
With the widespread deployment of Multimodal Large Language Models (MLLMs) in social interaction, understanding and controlling their behavior under complex personality conditions is essential. This paper introduces explicit personality conditioning and establishes a systematic evaluation framework encompassing single-personality induction, multi-personality induction, and personality switching. Experiments show that personality induction improves image captioning performance but can impair performance on tasks requiring precise reasoning, such as visual question answering (VQA). Balancing and residual effects are observed during multi-trait composition and dynamic switching, indicating that model behavior is co-modulated by both previous and current personality constraints. Existing prompt-based personality induction methods show limited transferability to multimodal settings. Our work reveals the dynamic and complex nature of personality modeling in MLLMs and underscores the need for robust, tailored methods for personality induction and evaluation. The code will be released when the paper is accepted.
