Search papers, labs, and topics across Lattice.
This paper introduces TriageRA-CCF, an adaptive rank budgeting approach for medical large language models that utilizes source-side clinical confidence and coverage signals to optimize parameter efficiency in question answering. By dynamically activating a subset of LoRA rank channels based on the confidence and clinical coverage of each question, the method addresses the limitations of fixed low-rank budgets that fail to account for variability in question difficulty. The results demonstrate that TriageRA-CCF outperforms existing baselines, achieving improved accuracy on both Qwen3-8B and Llama3.1-8B models, albeit with modest gains that vary across benchmarks.
TriageRA-CCF reveals that leveraging source-side clinical signals can significantly enhance the performance of medical LLMs by optimizing rank budgeting dynamically.
Medical large language models are commonly adapted with a fixed low-rank budget, even though medical questions differ substantially in confidence, clinical coverage, and cross-domain difficulty. We study adaptive rank budgeting for parameter-efficient medical question answering: for each question, the adapter decides whether to activate a small, medium, or large subset of LoRA rank channels. The central challenge is that a naive adaptive budget router can collapse to unstable choices or spend capacity without improving shifted benchmarks. We propose TriageRA-CCF, a source-side teacher for adaptive rank-budgeted LoRA. It combines three signals computed only from source training data: base-model answer confidence, metadata-cell clinical coverage, and a counterfactual close-miss proxy. These signals supervise a straight-through budget router over active ranks {2,4,8}, together with budget-cost, entropy, and rank-balance regularization. Under a matched CMB-source training protocol, TriageRA-CCF achieves the best average accuracy among LoRA, DoRA, and MoELoRA baselines on both Qwen3-8B and Llama3.1-8B. The gains are modest and non-uniform across benchmarks: +0.21 average points over the strongest external baseline on Qwen3-8B and +0.16 on Llama3.1-8B. Component ablations show that confidence, coverage, and counterfactual signals all provide useful budget supervision, but their combination is not monotonically best on every backbone.
Meta AI