Search papers, labs, and topics across Lattice.
This paper introduces FOCUS, a novel method that decouples expert personas in large language models (LLMs) to enhance their performance in domain-specific tasks. By employing orthogonal decomposition to extract and control expert persona vectors, FOCUS mitigates the issues of cross-domain coupling that can lead to inappropriate model behaviors in sensitive areas like healthcare and finance. Experimental results demonstrate that FOCUS significantly improves task accuracy across various domains compared to existing persona control techniques.
Decoupling expert personas in LLMs can drastically improve their accuracy and appropriateness in high-stakes domains like healthcare and finance.
Large Language Models (LLMs) can exhibit diverse personas, and activating expert personas has been shown to improve domain expertise and task accuracy. However, existing persona control methods often suffer from cross-domain coupling, which may lead to overly aggressive behavior in high-caution domains such as healthcare, or excessive conservatism in risk-sensitive domains such as financial trading. To address this issue, we propose FOCUS (\textbf{\underline{F}}ine-tuning with \textbf{\underline{O}}rthogonal \textbf{\underline{C}}ontrol for \textbf{\underline{U}}ncoupled persona\textbf{\underline{S}}). FOCUS first automatically extracts expert persona vectors from LLMs, then applies orthogonal decomposition to decouple domain-specific expert personas, and finally introduces an expert gating module to adaptively control persona activation according to task contexts. With a two-stage training strategy and a gated selection regularizer, the model learns to activate appropriate personas for both single-domain and cross-domain tasks. Experiments on financial, legal, medical, and cross-domain benchmarks show that FOCUS improves task accuracy and outperforms existing persona control methods. Our code is available at \href{https://anonymous.4open.science/r/openpersona-48F4}{this url}.
