Search papers, labs, and topics across Lattice.
To overcome the short-sightedness of single-turn RLHF in complex dialogues, SocialRL implements a multi-turn PPO framework that credit-assigns delayed outcome rewards alongside six fine-grained process rewards governing the goal-relationship trade-off. The approach uses dynamically generated rubrics and a stage-aware weight schedule that transitions agent focus from rapport-building early on to goal execution and balanced closure in later turns. Across varied synthetic and real social benchmarks, the method yields an average 9.2 percentage point increase in Goal Achievement over base LLMs without sacrificing conversational attunement.
Standard single-turn RL produces socially myopic agents, but dynamically scheduling rewards from early relationship-building to mid-dialogue task execution drives a 9.2 percentage point gain in multi-turn goal achievement.
Social intelligence enables agents to read social context, infer intent, and adapt over sustained dialogue. As language models become autonomous collaborators, it is central to building effective and trustworthy human-AI interaction. Existing reinforcement learning methods optimize single-turn utterances and sparse outcome rewards, producing short-sighted policies that struggle to manage goal-relationship tensions across multi-turn interactions. We propose SocialRL, a multi-turn reinforcement learning framework addressing both challenges. First, we apply multi-turn reinforcement learning using PPO that propagates delayed outcome rewards back to each turn, enabling long-horizon planning. Second, we design six process reward dimensions capturing the goal-relationship trade-off, including goal advancement, relational attunement, contextual coherence, etc. A reward model dynamically generates fine-grained scoring criteria for each dimension, while a stage-aware weight schedule prioritizes relationship-building in early turns, goal advancement mid-way, and balanced closure late. Across multiple social-dialogue benchmarks, SocialRL improves Goal Achievement by an average of 9.2 percentage points over the corresponding Base models. These results demonstrate the effectiveness of SocialRL across synthetic and real social scenes, as well as standard and challenging social scenarios.
