Search papers, labs, and topics across Lattice.
AttnDiff, a white-box fingerprinting framework, is introduced to verify if a suspect LLM is derived from a victim model, even after laundering operations. It uses minimally edited prompt pairs to induce semantic conflicts, captures differential attention patterns, and compares models using Centered Kernel Alignment (CKA) on spectral descriptors of these patterns. Experiments across Llama-2/3 and Qwen2.5 (3B-14B) show AttnDiff achieves high similarity scores for related derivatives ($>0.98$) and low scores for unrelated models ($<0.22$) with as few as 60 probes, enabling practical provenance verification.
LLMs leak uniquely identifiable fingerprints in their attention patterns when processing semantically conflicting prompts, enabling robust provenance verification even after fine-tuning, pruning, or merging.
Protecting the intellectual property of open-weight large language models (LLMs) requires verifying whether a suspect model is derived from a victim model despite common laundering operations such as fine-tuning (including PPO/DPO), pruning/compression, and model merging. We propose \textsc{AttnDiff}, a data-efficient white-box framework that extracts fingerprints from models via intrinsic information-routing behavior. \textsc{AttnDiff} probes minimally edited prompt pairs that induce controlled semantic conflicts, captures differential attention patterns, summarizes them with compact spectral descriptors, and compares models using CKA. Across Llama-2/3 and Qwen2.5 (3B--14B) and additional open-source families, it yields high similarity for related derivatives while separating unrelated model families (e.g., $>0.98$ vs.\ $<0.22$ with $M=60$ probes). With 5--60 multi-domain probes, it supports practical provenance verification and accountability.
