Search papers, labs, and topics across Lattice.
This paper investigates the challenges of optimizing prompts for LLM judges across multiple evaluation criteria, revealing significant performance degradation when gradients are combined. By extending the TextGrad method to handle multi-objective settings, the authors demonstrate a 59% drop in task-focus when feedback is provided on multiple criteria simultaneously. Additionally, they identify two distinct failure modes—optimization-time gradient dilution and inference-time instruction interference—that limit the effectiveness of multi-objective prompt optimization.
Multi-objective prompt optimization for LLM judges suffers from a staggering 59% drop in task-focus when gradients are combined, revealing critical design constraints.
Customizing an LLM judge to a specific problem or domain often involves optimizing its prompt across multiple evaluation criteria simultaneously. Textual gradient methods automate this for a single judge criterion, however they produce natural-language critiques, not numerical vectors. Thus, the conflict-resolution toolkit of multi-task learning (PCGrad, MGDA) does not apply to this multi-objective textual gradient setting. We extend TextGrad to the multi-objective setting and test four decomposition modes of textual gradient optimizers by varying how much cross-objective information the loss, gradient and optimizer LLMs share. We find the gradient's task-focus drops by 59% (9.0 to 3.7 out of 10) when the gradient LLM must provide feedback on multiple criteria jointly. Separately, we observe that naively combining single-objective optimized instructions into a single prompt degrades Spearman rho from 0.305 to 0.220 (-0.085). These results identify two separable failure modes: optimization-time gradient dilution and inference-time instruction interference, which together constrain the design space for multi-objective judge optimization using textual feedback.
Amazon Science