Search papers, labs, and topics across Lattice.
This paper analyzes trajectory matching for dataset condensation, revealing a representational bottleneck when using spectrally broad SGD trajectories to supervise synthetic data. To address this, they introduce Bezier Trajectory Matching (BTM), which replaces SGD trajectories with optimized quadratic Bezier curves between initial and final model states, reducing storage and structuring the supervision signal. Experiments on five clinical datasets show BTM matches or outperforms standard trajectory matching, especially in low-data regimes, suggesting structured supervision is key for effective condensation.
Ditching noisy SGD trajectories for smooth Bezier curves unlocks better dataset condensation, especially when data is scarce.
Dataset condensation constructs compact synthetic datasets that retain the training utility of large real-world datasets, enabling efficient model development and potentially supporting downstream research in governed domains such as healthcare. Trajectory matching (TM) is a widely used condensation approach that supervises synthetic data using changes in model parameters observed during training on real data, yet the structure of this supervision signal remains poorly understood. In this paper, we provide a geometric characterisation of trajectory matching, showing that a fixed synthetic dataset can only reproduce a limited span of such training-induced parameter changes. When the resulting supervision signal is spectrally broad, this creates a conditional representability bottleneck. Motivated by this mismatch, we propose Bezier Trajectory Matching (BTM), which replaces SGD trajectories with quadratic Bezier trajectory surrogates between initial and final model states. These surrogates are optimised to reduce average loss along the path while replacing broad SGD-derived supervision with a more structured, lower-rank signal that is better aligned with the optimisation constraints of a fixed synthetic dataset, and they substantially reduce trajectory storage. Experiments on five clinical datasets demonstrate that BTM consistently matches or improves upon standard trajectory matching, with the largest gains in low-prevalence and low-synthetic-budget settings. These results indicate that effective trajectory matching depends on structuring the supervision signal rather than reproducing stochastic optimisation paths.
