Search papers, labs, and topics across Lattice.
This paper identifies critical inefficiencies in existing dataset distillation methods that rely on a dual-compression process, which leads to significant information loss and unreliable label generation. The authors introduce CIM, a novel framework that directly minimizes the information gap between original and synthetic datasets, resulting in high-fidelity information condensation. Extensive experiments show that CIM achieves a new state-of-the-art performance on ImageNet-1K, distilling the dataset in just 80 minutes while surpassing previous methods by notable margins in accuracy.
A novel framework achieves unprecedented dataset distillation speed and accuracy by directly minimizing information loss, setting a new benchmark in the field.
Recent advancements in scaling dataset distillation rely heavily on decoupled information extraction pipelines, comprising SQUEEZE, RECOVER, and RELABEL stages. Despite their scalability to large-scale datasets, these methods suffer from prohibitive computational overhead and poor cross-architecture generalization. In this paper, we reveal the root cause of these bottlenecks: the implicit dual-compression process, from data to model and back to images, inherently induces severe information loss. Crucially, we empirically and theoretically demonstrate that this loss creates a distribution shift that fundamentally compromises the widely adopted RELABEL strategy, transforming the pre-trained model into an unreliable labeler that yields sub-optimal labels. To overcome these critical flaws, we propose CIM, a novel, metric-driven framework that abandons the flawed dual-compression paradigm. Instead, CIM explicitly quantifies and minimizes the information gap between the original and synthetic datasets. By directly aligning the data distributions, our approach ensures high-fidelity information condensation and inherently satisfies the prerequisites for effective relabeling. Extensive experiments demonstrate that CIM establishes a new state-of-the-art. Notably, it distills ImageNet-1K at an IPC=10 in merely 80 minutes on a single RTX-4090 GPU, achieving an unprecedented 48.7% Top-1 accuracy on ResNet-18 and significantly outperforming previous SOTA approaches, such as NRR-DD and DELT, by 2.6% and 2.9%, respectively. Our code is available at https://github.com/LINs-lab/CIM.
