Search papers, labs, and topics across Lattice.
This paper introduces a lightweight Bangla medical entity recognition (MedER) framework, compressing a 12-layer BanglaBERT teacher model into a 4-layer student network via Knowledge Distillation (KD) and INT8 quantization. The student learns from the teacher's pre-CRF soft emission logits, maintaining performance while significantly reducing computational cost. The final quantized student achieves an 8.6x CPU speedup and 48% storage reduction compared to the teacher.
Achieve 8.6x faster Bangla medical entity recognition on CPU while shrinking model size by 48% using knowledge distillation and quantization.
MedER refers to the identification of medical entities. It is crucial for extracting structured clinical information from unstructured medical text. Many existing systems rely on transformer-based models, which are computationally expensive and difficult to deploy in resource-constrained environments. Furthermore, earlier works often use relaxed evaluation metrics that artificially inflate performance by rewarding correct prediction of dominant "Outside" (O) tokens. In this paper, we propose a lightweight Medical Entity Recognition (MedER) framework for the Bangla language. We establish a rigorous baseline using a 12-layer BanglaBERT model combined with a Conditional Random Field (CRF) layer for exact-boundary entity detection. To address deployment constraints, we compress this teacher model into a 4-layer student network through Knowledge Distillation (KD), where the student learns from the teacher's pre-CRF soft emission logits. Finally, we apply INT8 dynamic quantization to further reduce model size and inference cost. Our final quantized student achieves an 8.6x CPU speedup while requiring nearly 48 percent less storage than the CRF teacher model.
