Search papers, labs, and topics across Lattice.
The paper introduces Quecto-V1, a 124M parameter GPT-2 based Small Language Model (SLM) trained from scratch on Indian legal statutes to address the resource divide in legal intelligence. It demonstrates the feasibility of on-device legal retrieval by applying 8-bit post-training quantization (GGUF format), achieving a model size under 150MB. Empirical results show Quecto-V1 outperforms general-purpose SLMs in domain-specific exact match retrieval tasks with minimal accuracy degradation after quantization, enabling offline deployment on consumer CPUs.
Domain-specific training and aggressive quantization lets a 124M parameter language model outperform larger, general-purpose models on legal retrieval tasks while running entirely offline on consumer CPUs.
The rapid proliferation of Large Language Models (LLMs) has revolutionized Natural Language Processing (NLP) but has simultaneously created a"resource divide."State-of-the-art legal intelligence systems typically rely on massive parameter counts (7B+) and cloud-based inference, rendering them inaccessible to practitioners in resource-constrained environments and posing significant data sovereignty risks. This paper introduces Quecto-V1, a domain-specific Small Language Model (SLM) engineered to democratize access to Indian legal intelligence. Built upon a custom configuration of the GPT-2 architecture (124 million parameters), Quecto-V1 was trained from scratch exclusively on a corpus of Indian statutes, including the Indian Penal Code (IPC), the Code of Criminal Procedure (CrPC), and the Constitution of India. Unlike generalist models, which prioritize broad world knowledge, our approach maximizes"lexical density"within the legal domain. Furthermore, we address the deployment bottleneck by applying post-training 8-bit quantization (GGUF format), compressing the model to a memory footprint of under 150 MB. Our empirical analysis demonstrates that Quecto-V1 achieves high fidelity in retrieving statutory definitions and penal provisions, outperforming general-purpose SLMs in domain-specific exact match tasks while running entirely offline on consumer-grade CPUs. We further present an ablation study showing that 8-bit quantization yields a 74% reduction in model size with less than 3.5% degradation in retrieval accuracy compared to full-precision baselines. These findings suggest that for specialized, high-stakes domains like law, domain-specific training coupled with aggressive quantization offers a viable, privacy-preserving alternative to monolithic cloud models.
