Search papers, labs, and topics across Lattice.
This study investigates the ability of large language models (LLMs) to infer cultural context and adapt their responses accordingly, revealing a significant gap between cultural knowledge and practical application. Using the newly introduced Cultural and Pragmatic Response Inference (CAPRI) dataset, the authors demonstrate that while LLMs can recognize cultural cues and recall relevant conventions, they often fail to apply this knowledge in generating culturally appropriate responses unless explicitly prompted. The findings indicate that LLMs' responses are influenced by their training data, leading to biases that reflect the dominant cultures of their origin, which could hinder effective communication across diverse cultural contexts.
LLMs can identify cultural context but often miss the mark in applying it, revealing a critical disconnect in culturally adaptive language generation.
Recent work has shown that LLMs overrepresent dominant cultures, particularly Western ones, while marginalizing others. We investigate whether this affects models' ability to generate culturally adapted responses by evaluating their use of local measurement units based on the user's perceived cultural background. We introduce Cultural and Pragmatic Response Inference (CAPRI), a dataset of conversations with varying levels of cultural cues. Experiments with state-of-the-art LLMs show that models can infer cultural background and recall relevant conventions, but often fail to utilize the information to adapt their answers to the relevant cultural conventions, unless explicitly prompted to perform the tasks sequentially. We further evaluate adaptation to the interpretation of time and quantity expressions, two subjective language grounding dimensions that are affected by culture. We find that models increasingly adapt their answers as cultural cues accumulate, but their priors are not culture-neutral, sometimes aligning with the model's country of origin. Overall, CAPRI provides a resource for future research aimed at narrowing the gap between cultural knowledge and culturally adaptive language generation.
