Search papers, labs, and topics across Lattice.
The paper introduces compact spectral circulant and block-circulant-with-circulant-blocks (BCCB) layers, which are FFT-diagonalizable circular convolutions parameterized directly in the real FFT domain, for memory-efficient and uncertainty-aware neural networks. By imposing spectral structure in the frequency domain and placing independent complex Gaussians on the Hermitian support, the authors induce an exact stationary Gaussian-process prior over filters, enabling structured variational inference and inexpensive Lipschitz bound calculations. Empirical results demonstrate that these spectral circulant/BCCB layers achieve competitive performance with significantly fewer parameters and tighter Lipschitz certificates in Bayesian neural networks, variational heads, and deterministic ViT projections.
Achieve comparable performance to strong baselines with spectral circulant layers while drastically reducing parameters and enabling tighter Lipschitz certificates.
Critical applications in areas such as medicine, robotics and autonomous systems require compact (i.e., memory efficient), uncertainty-aware neural networks suitable for edge and other resource-constrained deployments. We study compact spectral circulant and block-circulant-with-circulant-blocks (BCCB) layers: FFT-diagonalizable circular convolutions whose weights live directly in the real FFT (RFFT) half (1D) or half-plane (2D). Parameterizing filters in the frequency domain lets us impose simple spectral structure, perform structured variational inference in a low-dimensional weight space, and calculate exact layer spectral norms, enabling inexpensive global Lipschitz bounds and margin-based robustness diagnostics. By placing independent complex Gaussians on the Hermitian support we obtain a discrete instance of the spectral representation of stationary kernels, inducing an exact stationary Gaussian-process prior over filters on the discrete circle/torus. We exploit this to define a practical spectral prior and a Hermitian-aware low-rank-plus-diagonal variational posterior in real coordinates. Empirically, spectral circulant/BCCB layers are effective compact building blocks in both (variational) Bayesian and point estimate regimes: compact Bayesian neural networks on MNIST->Fashion-MNIST, variational heads on frozen CIFAR-10 features, and deterministic ViT projections on CIFAR-10/Tiny ImageNet; spectral layers match strong baselines while using substantially fewer parameters and with tighter Lipschitz certificates.
