Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
ELMOD is a 2.7B German language model optimized for on-device deployment, achieving high performance on resource-constrained hardware. The model was trained on a limited budget using innovative German-specific data preprocessing techniques that address unique linguistic features, alongside a quality filtering and rephrasing step to enhance instructional quality. As a result, ELMOD outperforms other models in its size class, rivaling the capabilities of larger 7B-parameter models in German language tasks.
ELMOD matches the performance of 7B models while being a fraction of the size, revolutionizing mobile language processing for German.
We present ELMOD - Efficient Language Model for On-Device Deployment - a compact (2.7B) German language model designed for efficient inference on resource-constrained hardware. ELMOD was trained on a limited computational budget (55k H100 GPU hours) using exclusively publicly available data. We developed a suite of German-specific data pre-processing, which differ from English-oriented counterparts in their handling of morphological variation, compounding, and orthographic conventions. Furthermore, we introduced a quality filtering and rephrasing step, which increased the instructional quality of the data, improved performance during the annealing phase, and reduced overall compute requirements. Thanks to our architectural model and data choices, including prefiltering, our educational-quality filtering and rephrasal to raise the educational-quality, ELMOD is the strongest performer in its size class (<3B), matching the performance of 7B-parameter models in German.