Search papers, labs, and topics across Lattice.
The authors construct FACTPROP, an entity-linked Wikipedia fact graph, to evaluate how fine-tuning-based knowledge updates corrupt previously mastered facts across realistic relational structures. Contrary to established findings on long-tail fragility during learning, correctly encoded facts associated with highly connected entities are the most vulnerable to collateral damage and propagate errors the furthest across the network. Leveraging this structural vulnerability, they develop PopAnchor, a targeted rehearsal strategy that anchors high-degree facts to significantly reduce downstream forgetting.
While long-tail knowledge is notoriously hard for LLMs to retain, structurally popular facts suffer the worst collateral damage during knowledge updates and act as super-spreaders of downstream hallucinations.
Updating a language model's knowledge through fine-tuning is essential for keeping its outputs current, yet can also induce factual forgetting and new hallucinations. Prior work shows that long-tail knowledge is harder to acquire and newly memorized long-tail facts are difficult to retain during later fine-tuning. We study a complementary question: among facts that a model has encoded correctly, which are most vulnerable to collateral corruption during other updates? To investigate this question under a realistic factual distribution, we construct a large-scale graph FACTPROP of verified Wikipedia facts by linking triples that share head or tail entities, thereby preserving connections among factual knowledge. We fine-tune models on factual statements and measure correct-to-incorrect facts after each update. Our results reveal a pattern distinct from prior findings on long-tail vulnerability during acquisition and retention: among facts that models already answer correctly, those associated with highly connected entities are more likely to be corrupted by neighboring updates, and updates to such facts propagate errors more broadly. Structural popularity therefore predicts both vulnerability and downstream damage. Inspired by this finding, we propose Popularity-based Anchoring (PopAnchor), a lightweight rehearsal strategy that preserves a small set of popular facts and reduces forgetting.
MIT CSAIL