Search papers, labs, and topics across Lattice.
This paper critically examines the effectiveness of parameter-level defenses against model merging, revealing that these defenses fail due to the dominance of pretrained model weights. By introducing the Anchor-Guided Attack (AGA), the authors demonstrate how attackers can exploit the small magnitude of protected task vectors to recover transformation matrices and bypass defenses. The study also proposes a countermeasure, Anchor-Repulsive Fine-tuning (ARF), which successfully mitigates the vulnerabilities exposed by AGA in empirical evaluations.
Parameter-level defenses against model merging are fundamentally flawed, allowing attackers to exploit their weaknesses with a new Anchor-Guided Attack.
The training-free integration of expert models via model merging has exposed significant security risks, enabling free-riders to combine specialized models without authorization. Recent works propose parameter-level defenses that employ linear parameter transformations to neutralize this threat. In this paper, we systematically analyze such defenses and reveal that their protected task vectors are inherently small in magnitude. Consequently, the protected weights remain overwhelmingly dominated by the pretrained model. Based on this observation, we designate the pretrained model as a static reference anchor and propose the Anchor-Guided Attack (AGA) to circumvent existing safeguards. Specifically, AGA aligns the protected model with this anchor to recover the transformation matrix analytically. Extensive evaluations validate that AGA consistently bypasses both individual and composite defenses under realistic defense-agnostic scenarios. Furthermore, we provide Anchor-Repulsive Fine-tuning (ARF), a defense method to mitigate the anchor dominance leveraged by AGA. Empirical results confirm that ARF effectively defeats the proposed attack. Our code is available at https://github.com/krumpguo/secure-merge-attack.
