Search papers, labs, and topics across Lattice.
This paper investigates the "Knowing--Using Gap" in fine-tuned large language models (LLMs), where memorized knowledge fails to translate into effective reasoning. By employing a novel intervention technique called self-patching, the authors analyze the internal dynamics of knowledge representation and identify critical activation locations that, when adjusted, significantly enhance generalization performance. Their findings reveal that while LLMs can memorize new information, misalignment in knowledge routing within the model's architecture leads to substantial generalization failures, which can be partially mitigated through a proposed heuristic strategy that recovers a significant portion of the potential performance gain.
Memorized knowledge in LLMs can exist without being effectively utilized, leading to a surprising 58–75% recovery in generalization performance through targeted internal adjustments.
Fine-tuning LLMs to inject new knowledge faces a critical challenge: LLMs can quickly memorize new facts, yet fail to use them for downstream reasoning tasks. We formalize this failure as the \textit{\textbf{Knowing--Using Gap}}, characterized by an accuracy gap and a temporal lag between memorization and generalization. To understand this phenomenon, we fine-tune LLMs with unseen knowledge and monitor the spatial permeation dynamics of the knowledge internally using a novel intervention technique called self-patching. Self-patching identifies activation locations where relocating representations substantially improves failed generalization cases. These results are consistent with a knowledge-circuit misalignment hypothesis: memorized representations can exist internally but may not be routed to computation-effective layers. To demonstrate the practicality of this diagnostic finding, we design a simple heuristic strategy which recovers 58--75\% of the oracle headroom in generalization failure. Experiments are done cross-domain for the robustness of this finding.
