Search papers, labs, and topics across Lattice.
This paper introduces Feedback-Aware Credit Assignment (FACA), a novel approach that enhances interactive reinforcement learning by incorporating local evidence from user reactions in multi-turn dialogues. By aligning each agent's reaction with preceding user interactions, FACA provides a more nuanced credit assignment that significantly improves performance in multi-domain tasks, achieving up to 10.22 percentage points better results compared to a traditional outcome-only method. The findings highlight the importance of leveraging immediate user feedback to refine agent learning and decision-making in dynamic environments.
Next-turn user reactions can boost multi-turn agent performance by over 10 percentage points, revealing the critical role of local feedback in reinforcement learning.
User-facing tool agents must coordinate dialogue and tool use as user goals unfold over multiple turns. Yet interactive reinforcement learning typically reduces each rollout to a terminal reward, assigning the same credit to effective elicitation, errors, and later repair. The next user turn is more than context: it also provides noisy, temporally local evidence about the preceding user-to-user segment. We introduce \textbf{F}eedback-\textbf{A}ware \textbf{C}redit \textbf{A}ssignment (\textsc{FACA}), which aligns each reaction with that segment, derives a locally normalized reaction advantage, and adds it to verified terminal outcome advantage without an extra critic or rollout. Against an outcome-only Interactive GRPO control matched in simulator, visible dialogue, initialization, rollout, and optimization, \textsc{FACA} improves the nine-domain $蟿$-family average across three independently trained runs by 5.91 and 10.22 percentage points at 8B and 14B, respectively. Gains concentrate in Telecom; at 8B, randomizing reaction polarity removes the Telecom gain. The same ordering holds zero-shot on Pare-Bench and Co-Gym. These results demonstrate that next-turn user reactions provide actionable local credit for improving multi-turn user-interacting agents.
