Search papers, labs, and topics across Lattice.
This paper investigates the challenges of merging differentially private task models, identifying two key geometric obstacles: local sharpness and reference drift, which hinder effective model integration. To address these issues, the authors introduce DP-Merging, a geometry-aware framework that employs a sharpness-aware objective and a reference-based alignment regularizer to enhance the mergeability of private models. Experimental results demonstrate that DP-Merging significantly improves the performance of merged models across various tasks while maintaining the privacy guarantees of the original fine-tuning processes.
Merging differentially private models can be dramatically improved by addressing geometric obstacles, leading to better performance without sacrificing privacy.
Model merging promises to construct a single multi-task model from independently fine-tuned task models without accessing the original task data. This makes it attractive when task data cannot be centralized, but released task models may still leak private fine-tuning data. Differential privacy (DP) provides a principled mechanism for limiting such leakage, yet its effect on model merging remains poorly understood. In this paper, we study the geometry of differentially private model merging and identify two geometric obstacles that make private task models difficult to merge: \emph{local sharpness}, which makes task losses sensitive to the parameter displacement induced by merging, and \emph{reference drift}, which measures the displacement of private task models from the shared pretrained initialization and amplifies cross-task interference. Based on these observations, we propose \textbf{DP-Merging}, a geometry-aware framework that improves the mergeability of differentially private task models. DP-Merging uses a DP-compatible sharpness-aware objective to guide each private task model toward flatter loss regions, and a reference-based alignment regularizer to keep task models close to the shared pretrained initialization. We derive a merge-gap upper bound showing that reducing local curvature and reference drift tightens the bound on the loss increase induced by merging. Experiments on vision and language tasks across multiple privacy budgets show that DP-Merging consistently improves private merged-model performance while preserving the privacy guarantees of the underlying DP fine-tuning procedures.
