Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces PALM, an algorithm for generating a small portfolio of LLMs that effectively covers a diverse range of user preferences, modeled as multi-dimensional weight vectors across traits like safety and humor. PALM provides theoretical guarantees on both the size and approximation quality of the LLM portfolio, balancing system cost and personalization. Experiments validate these guarantees and show improved output diversity compared to baselines.
Forget training a custom LLM for every user – a small, carefully chosen portfolio of LLMs can capture diverse preferences with provable guarantees.
The holy grail of LLM personalization is a single LLM for each user, perfectly aligned with that user's preferences. However, maintaining a separate LLM per user is impractical due to constraints on compute, memory, and system complexity. We address this challenge by developing a principled method for selecting a small portfolio of LLMs that captures representative behaviors across heterogeneous users. We model user preferences across multiple traits (e.g., safety, humor, brevity) through a multi-dimensional weight vector. Given reward functions across these dimensions, our algorithm PALM (Portfolio of Aligned LLMs) generates a small portfolio of LLMs such that, for any weight vector, the portfolio contains a near-optimal LLM for the corresponding scalarized objective. To the best of our knowledge, this is the first result that provides theoretical guarantees on both the size and approximation quality of LLM portfolios for personalization. It characterizes the trade-off between system cost and personalization, as well as the diversity of LLMs required to cover the landscape of user preferences. We provide empirical results that validate these guarantees and demonstrate greater output diversity over common baselines.