Search papers, labs, and topics across Lattice.
This paper explores the dual role of AI in the Indian subcontinent, highlighting its potential to both enhance access to linguistic resources and risk homogenizing diverse cultural perspectives. Through a longitudinal survey of the evolution of Natural Language Processing (NLP) in the region, the authors analyze the unique challenges posed by Indian languages, including their rich morphology and dialectal variation. The study culminates in the introduction of 'Culture Sensing', a research direction aimed at ensuring equitable AI performance across low-resource languages while producing culturally relevant outputs.
AI could either bridge or widen the gap in cultural representation, depending on how we approach Indic NLP development.
As Artificial Intelligence (AI) makes inroads into different parts of the Indian subcontinent, there is significant interest in studying how AI impacts the linguistic and cultural foundations of this civilization. AI is seen as a''double-edged sword''where on the one hand, it can enable access and inclusion for a large population, on the other, it can homogenize worldviews and exclude underrepresented languages and worldviews. In this paper, we try to characterize this problem by addressing the extensive characteristic nature of Indian linguistics and the way they closely connect to cultural practices and worldview. We then perform a longitudinal survey of how Natural Language Processing (NLP) techniques have evolved in this space, tracing the historical development of Indic NLP, covering key milestones, methodological shifts, and resource creation efforts. In addition, the paper also examines the structural and sociolinguistic characteristics of Indian languages, such as rich morphology, complex scripts and grammar rules, diglossia, and large dialectal variation, and explains how these create unique challenges for building AI foundation models. We then discuss the growing role of Indic foundation models and analyze how these models address these long-standing resource and representation gaps. Finally, we propose a research direction called'Culture Sensing', which re-imagines AI based on hermeneutic reasoning. Culture Sensing aims to address open problems such as ensuring equitable performance across low-resource languages and producing outputs that are culturally meaningful. By bringing together past work, current techniques, and emerging trends, this paper outlines research directions that can guide the next phase of Indic NLP and contribute to the development of more robust and inclusive Indic foundation models.
