Search papers, labs, and topics across Lattice.
This paper addresses the limitations of large language models (LLMs) in dynamic social interactions by introducing the Think-Strategy-Response (TSR) framework, which separates dialogue into strategic planning and linguistic execution. The authors employ Linearized Hierarchical Reinforcement Learning with Variance-Gated Rewards (LHRL-VGR) to optimize reward allocation based on goal achievement variance, allowing for more nuanced and effective learning. Experimental results on the SOTOPIA benchmark reveal that their approach improves a Qwen2.5-7B agent's goal completion success by 7.32% over the GPT-4o baseline, marking a significant advancement in multi-agent negotiation tasks.
LLMs can achieve 7.32% better goal completion in social negotiations by strategically optimizing reward signals based on dialogue context.
Large language models (LLMs) excel in structured tasks but struggle with dynamic social interactions, where success requires long-term goal coordination and rapid adaptation. Current methods often apply uniform goal-based rewards to every utterance, overlooking the specificity of objectives at each dialogue turn and failing to account for the rationale of potential strategies. Inspired by the Theory of Planned Behavior, we propose the Think-Strategy-Response (TSR) framework, which decomposes social dialogue into two hierarchical stages: high-level strategic planning and low-level linguistic execution. To optimize TSR, we introduce Linearized Hierarchical Reinforcement Learning with Variance-Gated Rewards (LHRL-VGR), a novel algorithm that dynamically routes rewards - balancing goal completion and strategy adherence - based on the variance of goal achievement scores. Experiments on the SOTOPIA benchmark show that our approach fine-tunes a Qwen2.5-7B agent to surpass the GPT-4o baseline by 7.32% in goal completion success, demonstrating state-of-the-art performance in multi-agent social negotiation tasks.
