Search papers, labs, and topics across Lattice.
This study investigates the "Knowing--Using Gap" in fine-tuned large language models (LLMs), where memorized knowledge does not translate into effective reasoning. By employing a novel intervention technique called self-patching, the authors identify specific activation locations that, when adjusted, significantly enhance generalization performance. The findings reveal that while LLMs can memorize new information, misalignment in knowledge routing prevents effective utilization, leading to a recovery of 58-75% of potential generalization improvement through targeted adjustments.
Memorized knowledge in LLMs can remain dormant due to misalignment in activation routing, but targeted interventions can unlock substantial gains in reasoning performance.
Fine-tuning LLMs to inject new knowledge faces a critical challenge: LLMs can quickly memorize new facts, yet fail to use them for downstream reasoning tasks. We formalize this failure as the \textit{\textbf{Knowing--Using Gap}}, characterized by an accuracy gap and a temporal lag between memorization and generalization. To understand this phenomenon, we fine-tune LLMs with unseen knowledge and monitor the spatial permeation dynamics of the knowledge internally using a novel intervention technique called self-patching. Self-patching identifies activation locations where relocating representations substantially improves failed generalization cases. These results are consistent with a knowledge-circuit misalignment hypothesis: memorized representations can exist internally but may not be routed to computation-effective layers. To demonstrate the practicality of this diagnostic finding, we design a simple heuristic strategy which recovers 58--75\% of the oracle headroom in generalization failure. Experiments are done cross-domain for the robustness of this finding.
