Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper investigates the synergistic effects of learnable downsampling and spectral-domain filter parameterization in convolutional networks, finding that spectral kernels benefit more from learned resolution allocation than spatial kernels. The authors systematically examine this interaction using cropped resizing and demonstrate consistent accuracy gains across three datasets and two model sizes, with spectral kernels showing a notable advantage even when total retained spatial area is matched. These findings highlight the distinct responses of spatial and spectral representations to dynamic resolution adjustments.
Spectral convolutional kernels unlock significantly greater accuracy gains from learned resolution allocation than their spatial counterparts, even when spatial area is normalized.
Learnable downsampling and spectral-domain filter parameterizations have each made convolutional networks more flexible, but their interaction has not been systematically examined. We bridge this gap by examining convolutional kernels learned in the spatial and spectral domains, together with learned downsampling ratios under a cropped resizing implementation. We find that spectral and spatial kernels respond differently to learned resolution allocation, with larger gains for the spectral kernel. We observe this consistently on three datasets (EuroSAT, CIFAR-10 and STL-10) and at two model sizes (differences in accuracy gain of +0.013 to +0.028 on held-out test sets). Part of this difference reflects the greater gain of the spectral kernel from retaining more spatial area, but the difference remains positive when total retained area is matched. This pattern is observed across datasets and model sizes, although its magnitude and dependence on the complexity budget vary with model size and the resizing implementation.