Search papers, labs, and topics across Lattice.
This paper investigates the feasibility of lineage verification for open-weight language models using only model weights, addressing the challenge of undocumented model provenance. By removing the shared identity-aligned component from residual training, the authors develop a symmetric lineage score that effectively distinguishes between fine-tuned, merged, and pruned models versus independent checkpoints. Their method achieves an impressive AUROC of 1.0 on various benchmarks, demonstrating its robustness and efficiency, running 76 times faster than existing baselines while maintaining accuracy in identifying model ancestry.
A lineage verification method that distinguishes model ancestry with perfect accuracy, even under aggressive checkpoint modifications.
Open-weight language models are fine-tuned, quantized, pruned, and merged, yet their provenance is often undocumented. We study data-free white-box lineage verification: can weights alone reveal whether two compatible model checkpoints share ancestry? Residual training produces a shared identity-aligned component in branch products, so this structure alone cannot establish ancestry. We remove it and compare checkpoint-specific structure across residual blocks, yielding a symmetric lineage score calibrated against independent checkpoints. On residual-MLP and GPT-2 benchmarks, the score separates fine-tuned, LoRA-merged, pruned, and quantized descendants from independent and distilled models (AUROC=1.0), distinguishing weight ancestry from behavioral similarity. Under function-preserving checkpoint laundering experiments, weight-space baselines lose margin or fail; our score remains unchanged and runs 76x faster than the nearest robust baseline on GPT-2. The projection-pairing signal appears across six language-model families and beyond, and a case study correctly identifies 3 related and 7 unrelated LLaMA-2 public checkpoints. Collectively, these results establish a passive, data-free provenance signal for compatible open-weight language-model checkpoints
