Search papers, labs, and topics across Lattice.
This paper introduces self-supervised representation-guided generative dataset distillation (SRG), a novel framework that leverages the geometry of self-supervised learning (SSL) representations to enhance dataset distillation. By constructing class-wise prototypes from SSL representations and employing a stage-wise guidance strategy during diffusion sampling, SRG effectively aligns generated samples with the discriminative features of pretrained models. The approach consistently outperforms existing generative baselines across various datasets, highlighting its potential for improving the efficiency of training data utilization in modern vision systems.
SRG transforms dataset distillation by aligning generative samples with the discriminative geometry of self-supervised representations, achieving superior performance across multiple benchmarks.
Dataset distillation compresses a large training set into a compact synthetic set while retaining its downstream utility. Most existing methods target randomly initialized networks, whereas modern vision systems often adapt frozen pretrained encoders with lightweight modules. Distilled samples should therefore preserve the discriminative geometry of the pretrained representation space, which existing generative objectives do not explicitly consider. We propose self-supervised representation-guided generative dataset distillation (SRG), a framework that translates the SSL geometry into diffusion guidance. Specifically, SRG constructs class-wise prototypes from real-image SSL representations and performs guidance through three SSL-space objectives for prototype alignment, inter-class discrimination, and intra-class assignment. During diffusion sampling, it adopts a stage-wise guidance strategy: early denoising is anchored to the latent of the real image whose SSL representation is nearest to the assigned prototype, whereas later denoising is guided by the SSL-space objectives. This division preserves the visual realism provided by the generative prior while progressively steering samples toward representative and class-discriminative regions of the SSL representation space. SRG consistently outperforms the evaluated generative baselines across multiple datasets and IPC settings. A cross-encoder evaluation further indicates transfer across pretrained representation spaces. These results demonstrate the effectiveness of representation-guided generation for dataset distillation with pretrained SSL models.
