Search papers, labs, and topics across Lattice.
This study presents Andha-Dhun, the first dataset of human-authored audio descriptions (ADs) in Hindi, collected from eight full-length movies, addressing the lack of ADs in Indian languages. The authors explore two generation methods: one that derives ADs from English dense video descriptions and another that translates existing English ADs into Hindi, revealing significant shortcomings in both approaches. Key findings indicate that naive translation leads to artifacts and reduced diversity, highlighting the necessity for culturally adapted content to enhance accessibility for Blind and Low Vision audiences in India.
Naive translation of audio descriptions fails to capture cultural nuances, revealing a critical gap in accessibility for India's Blind and Low Vision communities.
Audio Descriptions (ADs) narrate visual content for Blind and Low Vision (BLV) audiences during gaps in audiovisual media. There is growing momentum around ADs in movies and TV shows, and with mandates from India's Central Board of Film Certification (CBFC), there is a need to expand ADs beyond English. Yet, there is no work that generates ADs for any Indian language. To address this gap, we present the first systematic study of ADs in Hindi, contributing to aspects such as data, generation, and evaluation. We introduce Andha-Dhun, the first dataset of human-authored Hindi ADs collected from 8 full-length movies. We explore two approaches for generating ADs in Hindi: (i) directly from English dense video descriptions, and (ii) translating English ADs into Hindi. We evaluate these approaches using perplexity and LLM-as-a-judge metrics to assess fluency and quality respectively. We also analyze movies that have both English and Hindi human-authored ADs and find that naive translation introduces artifacts and narrows diversity compared to original Hindi ADs. Direct machine translation fails to adapt cultural references, while human-translated ADs do better but still fall short. Our findings emphasize that the purpose of Hindi ADs is accessibility for Indian BLV audiences, and that this requires adapting content for the audience more than strict fidelity to the source.
