Search papers, labs, and topics across Lattice.
This paper introduces Hard-Routed MoR-LoRA, a novel framework for integrating independently trained LoRA adapters into a single large language model using hard selection rather than soft mixtures. By employing a two-stage process that involves training domain-specific adapters with reinforcement learning and then freezing them for integration through a lightweight router, the method maintains expert performance while significantly reducing the number of trainable parameters. Experiments demonstrate that this approach not only preserves the reasoning capabilities of the experts but also reveals that softer routing methods often lead to inefficient concentration on a single expert, highlighting the advantages of hard routing.
Hard routing in LoRA expert composition can drastically reduce trainable parameters while preserving reasoning performance, challenging the efficacy of soft routing methods.
Composing independently trained LoRA adapters into a single large language model is useful for multi-domain adaptation, especially when the original training data cannot be shared. A common approach is to use MoE-style routing over LoRA experts, but for frozen pretrained adapters, soft weighted combinations can change the unit-scale additive update under which each LoRA module was originally trained. We propose \textbf{Hard-Routed MoR-LoRA}, a two-stage framework for composing frozen reasoning LoRA experts through unit-scale hard selection. First, domain-specific LoRA adapters are trained independently using reinforcement learning from verifiable feedback to obtain reasoning experts. Then, all experts are frozen, reasoning traces are distilled from them, and only a lightweight shared router together with a small attention LoRA is trained for integration. The router selects exactly one expert per token using hard top-1 routing, while a straight-through estimator enables gradient-based training. Experiments across five benchmarks, multiple model scales, and additional model families show that Hard-Routed MoR-LoRA preserves expert behavior while requiring substantially fewer trainable parameters than soft-routing mixture baselines. Our analysis further shows that normalized soft mixtures often concentrate most routing mass on a single expert, suggesting that hard unit-scale routing provides a simple and efficient abstraction for frozen LoRA expert composition.
