Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This study evaluates the robustness of event detection systems for Bangla in the presence of real-world noise, utilizing a new benchmark of 9,979 annotated sentences across various text types. The findings reveal that while encoder models like BanglaBERT and XLM-R perform better on clean text, they suffer significant performance drops under noisy conditions, whereas decoder-only models such as Llama 3 and Gemma 3 exhibit greater resilience to noise. Additionally, the research highlights that combining training on both clean and noisy data effectively enhances the robustness of encoder models, thereby narrowing the performance gap between the two architectures.
Decoder-only models significantly outperform encoder architectures in noisy conditions, challenging the prevailing preference for encoder-based systems in event detection tasks.
Event detection (ED) systems are typically evaluated on clean, curated text, leaving their robustness to real-world noise largely unexplored, particularly for low-resource languages such as Bangla. We introduce a generalized Bangla news event ontology and a benchmark comprising 9,979 annotated sentences across 40 event subtypes, spanning clean news text, real-world Automatic Speech Recognition (ASR) transcripts, and orthographically corrupted text. We systematically evaluate fine-tuned encoder-only models (BanglaBERT and XLM-R) alongside instruction-tuned decoder-only large language models (Llama 3 and Gemma 3). Our results reveal a clear architectural trade-off: encoder models achieve higher performance on clean text but degrade substantially under noise, whereas decoder-only LLMs are markedly more robust, particularly when event triggers are corrupted. We further show that embedding annotation guidelines during instruction tuning establishes a higher performance baseline on noisy text but yields inconsistent reductions in performance degradation across noisy conditions. Finally, model scaling consistently improves the robustness of decoder-only LLMs, while combined training on clean and noisy data serves as an effective regularization strategy that disproportionately benefits encoder architectures, significantly narrowing the robustness gap.