Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
The authors investigate the "peak-then-collapse" phenomenon in LLMs using a knowledge-graph tool with a minimal API (four Freebase navigation verbs) on the Complex WebQuestions dataset. They find that after an initial performance increase via RLVR, the tool-grounded answer rate collapses rapidly, and that reward redesigns do not eliminate this failure. Through ablation studies, they identify that the lack of informative feedback from the minimal API, unlike Python interpreters or web search, is a key factor, and that retrieval-composition failures are the dominant error source.
LLMs trained to use knowledge graphs can exhibit a "peak-then-collapse" behavior, where performance initially improves with RL but then catastrophically degrades, revealing a surprising fragility in tool use.
We test the standard RLVR tool-use recipe -- GRPO on Qwen2.5-7B-Instruct -- on a deliberately minimal knowledge-graph tool API: four Freebase navigation verbs over Complex WebQuestions. Under a self-verifiable retrieval reward, the policy's tool-grounded answer rate climbs from $3.8\%$ to $9.6\%$ over 250 steps, then collapses to $0\%$ within a single 50-step window -- a \emph{peak-then-collapse} pattern replicated across four seeds. Across seven reward designs, we find four recurring failure modes: adding denser or more targeted proxy rewards shifts the failure mode rather than eliminating it. We argue that a key difference from Python interpreters, web search, and JSON APIs is interface feedback: their failures often leak natural-language signal the model saw in pretraining. A Python traceback names the failing line; an empty Freebase result \texttt{[]} does not. Stripping away that surface exposes a degradation regime that same-family reward redesigns do not fix. A direct oracle ablation rules out relation selection: injecting gold relations at every retrieval call lifts exact-match accuracy by only $+0.20$~pp, and $95.4\%$ of retrieval-dependent errors are retrieval-composition failures rather than answer-extraction failures. As a mitigation, one-iteration self-distillation reaches $40.0\%$ EM at 7B and is capacity-invariant: doubling capacity to 14B improves EM by only $0.25$~pp, and initialization barely matters -- the ceiling appears interface-bound within the 7B--14B range tested.