Search papers, labs, and topics across Lattice.
This paper introduces the task of culture-aware humorous captioning, requiring models to generate humorous captions conditioned on images and specific cultural contexts. To facilitate this, the authors create a six-dimensional evaluation framework and a staged alignment framework. The staged alignment framework uses judge-based GRPO with a Degradation-aware Prototype Repulsion Constraint to improve contextual fit and balance image relevance with humor across cultures.
Current multimodal humor generation struggles to maintain image relevance, contextual appropriateness, and humor quality when explicitly conditioned on different cultural contexts, but this work offers a way to improve it.
Recent multimodal large language models have shown promising ability in generating humorous captions for images, yet they still lack stable control over explicit cultural context, making it difficult to jointly maintain image relevance, contextual appropriateness, and humor quality under a specified cultural background. To address this limitation, we introduce a new multimodal generation task, culture-aware humorous captioning, which requires a model to generate a humorous caption conditioned on both an input image and a target cultural context. Captions generated under different cultural contexts are not expected to share the same surface form, but should remain grounded in similar visual situations or humorous rationales.To support this task, we establish a six-dimensional evaluation framework covering image relevance, contextual fit, semantic richness, reasonableness, humor, and creativity. We further propose a staged alignment framework that first initializes the model with high-resource supervision under the Western cultural context, then performs multi-dimensional preference alignment via judge-based GRPO with a Degradation-aware Prototype Repulsion Constraint to mitigate reward hacking in open-ended generation, and finally adapts the model to the Eastern cultural context with a small amount of supervision. Experimental results show that our method achieves stronger overall performance under the proposed evaluation framework, with particularly large gains in contextual fit and a better balance between image relevance and humor under cultural constraints.
