Search papers, labs, and topics across Lattice.
This paper introduces $S^{2}$-FracMix, a novel data augmentation technique that generates label-consistent samples by utilizing multi-scale salient patches from the same image, thereby avoiding the semantic disruption common in traditional mixup methods. The approach enhances model robustness by incorporating self-similarity patterns into salient regions through an adaptive mixing scheme, FracMix, which promotes scale-invariant feature learning. Extensive evaluations across seven benchmarks demonstrate that $S^{2}$-FracMix achieves state-of-the-art performance in classification, robustness, calibration, object detection, and transfer learning tasks, highlighting its effectiveness in improving generalization of deep visual models.
Label-preserving self-saliency mixup can significantly boost model performance while maintaining semantic integrity in data augmentation.
Data augmentation is known to improve generalization of deep visual models. Recent methods favor mixup strategies that generate interpolated samples to improve model performance. However, these techniques not only incur significant computational overhead, they also lead to semantic disruption of augmentation data due to cross-sample mixing. We first propose Self-Saliency ($S^2$) Mixup, which constructs challenging yet label-consistent samples by extracting multi-scale salient patches and reinserting them into non-salient regions of the same image. This promotes scale-invariant feature learning while avoiding cross-sample interference. To further enhance model robustness, we introduce FracMix, a mixing scheme that injects self-similarity patterns into salient regions using adaptive ratios. Collectively, our unified framework, $S^{2}$-FracMix, enables simultaneous learning from fractal and non-fractal structures within a single image, yielding a targeted and structurally coherent augmentation strategy. We theoretically analyze the advantage of our technique, and empirically establish its superiority over the existing methods by achieving state-of-the-art performance in extensive evaluation with seven benchmarks across classification (coarse and fine-grained), robustness, calibration, object detection, and transfer learning tasks. Project page is available at \href{https://fracmix-data-augmentation.github.io/}{fracmix-data-augmentation.github.io}
