Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces MindRouter, an open-source LLM inference gateway that enables universities to effectively manage AI workloads by balancing cloud and on-premises systems to maintain institutional control. By supporting sensitive workloads and providing a unified API layer that integrates various client protocols, MindRouter allows institutions to retain sovereignty over data and infrastructure while ensuring compliance with regulations like FERPA. Deployed successfully at the University of Idaho, the system demonstrates that mid-sized institutions can operate self-hosted AI services efficiently, serving over 64 models on a 40 GPU cluster while implementing robust access controls and monitoring features.
Mid-sized universities can now maintain control over sensitive AI workloads without sacrificing scalability, thanks to MindRouter's innovative architecture.
As large language models become more integrated into research, teaching, and administration, universities must decide which AI workloads belong in the cloud and which require stronger institutional control. This is not an either-or choice: an effective AI strategy combines cloud and on-premises systems to preserve AI sovereignty where it matters most. In this context, AI sovereignty means retaining control over infrastructure, data, models, access policies, guardrails, monitoring, and governance so that sensitive or strategic workloads remain aligned with institutional requirements while enabling transparent energy accounting and lower costs at scale. This paper presents MindRouter, an open-source LLM inference gateway designed to help universities operate self-hosted AI services on institutional GPU infrastructure as part of a balanced AI strategy. MindRouter is designed to support sensitive workloads involving FERPA-regulated student data, controlled unclassified information (CUI), export-controlled data, and institutionally managed intellectual property (IP). The system provides a unified API layer that translates among OpenAI, Ollama, and Anthropic client protocols, routes requests across heterogeneous GPU backends using a Weighted Deficit Round Robin (WDRR) fair-share scheduler, and incorporates role-based access control, audit logging, configurable guardrails, and operational observability through real-time GPU telemetry and system-level monitoring. Deployed in production at the University of Idaho on a cluster of 40 GPUs serving more than 64 open-weight models, MindRouter shows that mid-sized institutions can operate institutionally managed AI services while retaining direct control over data, infrastructure, and operational policy. MindRouter is released under the Apache 2.0 license and is available at MindRouter.ai