Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces BanglaWild, a comprehensive benchmark for in-the-wild Bengali scene text recognition, comprising 2,535 images with detailed transcriptions and diagnostic attributes. The evaluation of fifteen vision-language models (VLMs) and three conventional OCR systems reveals that larger models do not consistently outperform smaller ones, with visual mis-recognition being the predominant source of errors. Additionally, the study highlights the effectiveness of LoRA in mitigating catastrophic failures in weaker models, while also demonstrating the impact of prompt language on performance across different scripts.
Larger models in Bengali scene text recognition fail to outperform their smaller counterparts, revealing a surprising reliance on visual recognition errors.
In-the-wild Bengali scene text recognition is largely unmeasured: existing resources target handwritten documents or constrained sign-board parsing, report only aggregate edit-distance metrics, and evaluate either conventional OCR or VLMs, never both on the same in-the-wild data. To address this gap, we introduce BANGLAWILD, a benchmark of 2,535 Bengali scene text images, each paired with a verbatim gold transcription, two categorical axes, four diagnostic attributes, and an orthographically standard form where the in-image text deviates from canonical spelling. We evaluate fifteen VLMs and three conventional OCR systems under three prompting strategies, fine-tune 6 open-source models with LoRA, and complement edit-distance metrics with an LLM-as-a-Judge evaluation. Our results reveal a persistent gap in which larger models within the same family do not outperform smaller ones. Our fifteen-class error taxonomy shows that visual mis-recognition accounts for ~60% of errors in the strongest systems, while conjunct-related errors contribute under 2%, challenging a long-standing assumption in Bengali OCR research; the same visual dominant profile also holds across architectures, including the one conventional baseline that reads Bengali reliably. Prompt language mainly affects cross-script drift and LoRA reduces catastrophic failures in weak models without lifting the ceiling on already competent ones. Code and data will be publicly released.