Search papers, labs, and topics across Lattice.
This paper investigates the effectiveness of partial data augmentation compared to full augmentation in machine learning, utilizing Fourier analysis and representation theory of finite groups. The authors demonstrate that for a wide range of classical learning problems, partial augmentation can achieve the same minimax rates as full augmentation, with the approximation error diminishing as the subset size increases. Additionally, they establish that exact invariance through data augmentation necessitates averaging over the entire group, highlighting the computational limitations of strict subsets in expressive hypothesis spaces.
Partial data augmentation can match the statistical benefits of full augmentation, challenging the notion that complete symmetry is necessary for optimal learning.
Data augmentation is a simple and model-agnostic approach for exploiting known invariances in learning problems. Given a group acting on the input space, one augments the training set with transformed copies of each sample. Because it exploits symmetries without modifying the underlying learning algorithm, data augmentation can be applied broadly across learning methods. However, this universality comes at a computational cost: when the group is large, full group-sized augmentation quickly becomes computationally infeasible. This raises a fundamental question: Can partial data augmentation achieve the same statistical benefits as full augmentation in terms of generalization and sample complexity? We develop a general framework for investigating this question using Fourier analysis and the representation theory of finite groups. We show that, for a broad class of classical learning problems, partial data augmentation based on a randomly sampled subset of group elements achieves the same minimax rates as full augmentation, up to an approximation error that vanishes as the subset size increases. Our results provide a theoretical explanation for why partial augmentation can retain the statistical benefits of full augmentation despite enforcing symmetry only approximately, and shed light on a recently raised question in learning with symmetries: whether statistically optimal learning under general group invariances can be achieved using computationally scalable methods. Moreover, we prove a complementary impossibility result: enforcing exact invariance via data augmentation requires averaging over the entire group, and cannot be achieved by any strict subset when the hypothesis space is sufficiently expressive. Together, these results provide a unified perspective on full and partial data augmentation, as well as exact and approximate symmetry enforcement.
