Search papers, labs, and topics across Lattice.
This paper introduces IR3DE, a Ridge Regression-based router designed to efficiently select the most suitable domain-expert large language model (LLM) for given prompts. Unlike existing methods that either focus on cost optimization or require extensive retraining, IR3DE offers rapid routing decisions while maintaining high performance, achieving a normalized score of 98.4% in reasoning tasks. The ability to dynamically add or remove domain experts without retraining the router enhances its practicality for real-world applications.
IR3DE achieves near-optimal performance in reasoning tasks while enabling seamless integration of new domain experts, all without the need for retraining.
Foundational Large Language Models (LLMs) demonstrate proficiency on a wide range of general tasks, and achieve remarkable results on various specialized tasks via domain-expert LLMs. With the ever-growing list of available LLMs, inference routers are being proposed to select the most appropriate LLM for each prompt. However, existing routing methods either optimize cost across weak-to-strong generalist LLMs or require substantial training to support domain-expertise routing. In this paper, we propose IR3DE, a Ridge Regression-based Router for Domain Experts that provides cheap and fast routing decisions for each prompt. We evaluate IR3DE in two Causal Language Modeling (CLM) settings where the tasks are next-token prediction for all domains, and one reasoning setting where each domain has its own distinct reasoning task. Despite being a linear router, IR3DE achieves performance comparable to the other baselines in both CLM settings, and surpassing them in the reasoning setting, with a normalized performance of 98.4%. Moreover, IR3DE enables the addition or removal of new domain experts without requiring the router to be retrained from scratch, allowing a dynamic set of LLMs to be served with minimal disruption to the router itself. Our code is available at: github.com/gensyn-ai/IR3DE.
