Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This study investigates the effectiveness of fine-tuning small language models for legal question answering (QA) in Bangladesh by incorporating relevant statutory provisions into the training process. By curating a dataset of 2,165 bilingual QA records from Bangladeshi legal texts and fine-tuning models of varying sizes (0.8B, 2B, and 4B), the authors demonstrate significant improvements in model performance, particularly with the 0.8B model, which achieved a score increase from 2 to 34 out of 100 on the 2022 English Bar Council exam. Notably, fine-tuning not only enhances accuracy but also drastically reduces language drift in responses, indicating that small models can effectively utilize the legal context provided to them.
Fine-tuning small models on relevant legal data can dramatically boost their accuracy in legal QA, with the 0.8B model showing a remarkable score increase of 1,600% on a key exam.
A small language model can receive the governing statutory provision and still answer incorrectly. We test whether fine-tuning on examples containing relevant law improves later use of retrieved law. We curate 2{,}165 bilingual QA records from six Bangladeshi acts and three schedules, then fine-tune Qwen3.5 at 0.8B, 2B, and 4B. Evaluation uses the 2022 and 2023 Bangladesh Bar Council exams in Bangla and machine-translated English, with no retrieval, BM25, or FAISS, scored by strict consistency over three seeded runs. At 0.8B, fine-tuning raises the 2022 English FAISS score from 2 to 34 of 100. Gains at 0.8B and 2B survive paired testing, but the 4B model has no detectable net gain: Bangla improves while several English conditions regress. Fine-tuning also reduces answers that drift from Bangla into mostly English from 44.0--53.2\% to 0.2--0.7\%, with adjusted $p<.001$ at every scale. Retrieval quality is therefore not the only bottleneck. Small bilingual legal models also differ in how they use supplied law and whether they answer in the requested language. The dataset is publicly available at https://huggingface.co/datasets/momahadi/bangladesh-legal-qa-dataset.