Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper surveys 47 credit assignment (CA) methods used in reinforcement learning for large language models, focusing on the challenges posed by reasoning RL (single chain-of-thought) and agentic RL (multi-turn interaction). The authors categorize these methods based on assignment granularity and methodology, revealing a shift in CA techniques as RL moves from reasoning to agentic tasks. Beyond the survey, the paper provides a machine-readable inventory of papers, a reporting checklist for future CA research, and a benchmark protocol specification to facilitate further progress.
Agentic RL is forcing a Cambrian explosion of new credit assignment techniques—hindsight counterfactual analysis, privileged asymmetric critics, and turn-level MDP reformulations—that have no precedent in reasoning RL.
Reinforcement learning (RL) for large language models (LLMs) increasingly relies on sparse, outcome-level rewards -- yet determining which actions within a long trajectory caused the outcome remains difficult. This credit assignment (CA) problem manifests in two regimes: reasoning RL, where credit must be distributed across tokens and steps within a single chain-of-thought generation (500--30K+ tokens); and agentic RL, where multi-turn environment interaction introduces stochastic transitions, partial observability, and horizons of 100+ turns (100K--1M tokens), making episode-level credit increasingly uninformative. We survey 47 CA methods (41 core, 6 adjacent enablers) published between 2024 and early 2026, organizing them in a two-dimensional taxonomy by assignment granularity (token, segment, step, turn, multi-agent) and methodology (Monte Carlo, temporal difference, model-based, game-theoretic, information-theoretic). Beyond the survey itself, we contribute three reusable resources: (1) a structured, machine-readable paper inventory with taxonomy labels, baseline families, and evidence levels; (2) a reporting checklist for future CA papers, validated against the reviewed literature to identify systematic methodological gaps; and (3) a benchmark protocol specification with task families, metadata requirements, and controlled bifurcation tasks, accompanied by a method selection decision tree. Our synthesis suggests that the shift from reasoning to agentic RL complicates and reshapes the credit assignment landscape: reasoning CA is maturing around process reward models and critic-free group comparison, while agentic CA is driving genuinely new approaches -- hindsight counterfactual analysis, privileged asymmetric critics, and turn-level MDP reformulations -- that have no direct precedent in reasoning RL.