Search papers, labs, and topics across Lattice.
The paper introduces Osmosis Distillation (OD), a novel model hijacking attack that uses poisoned, synthetically distilled datasets to compromise deep learning models during transfer learning. OD achieves high attack success rates on hidden tasks while maintaining utility on original tasks, using only a few poisoned samples. The attack's effectiveness is demonstrated across various datasets and model architectures, highlighting a vulnerability in transfer learning when using third-party synthetic datasets.
A shockingly small number of poisoned, synthetically distilled data points can completely hijack a model during transfer learning, turning it into an unwitting accomplice.
Transfer learning is devised to leverage knowledge from pre-trained models to solve new tasks with limited data and computational resources. Meanwhile, dataset distillation has emerged to synthesize a compact dataset that preserves critical information from the original large dataset. Therefore, a combination of transfer learning and dataset distillation offers promising performance in evaluations. However, a non-negligible security threat remains undiscovered in transfer learning using synthetic datasets generated by dataset distillation methods, where an adversary can perform a model hijacking attack with only a few poisoned samples in the synthetic dataset. To reveal this threat, we propose Osmosis Distillation (OD) attack, a novel model hijacking strategy that targets deep learning models using the fewest samples. Comprehensive evaluations on various datasets demonstrate that the OD attack attains high attack success rates in hidden tasks while preserving high model utility in original tasks. Furthermore, the distilled osmosis set enables model hijacking across diverse model architectures, allowing model hijacking in transfer learning with considerable attack performance and model utility. We argue that awareness of using third-party synthetic datasets in transfer learning must be raised.
