Search papers, labs, and topics across Lattice.
This paper introduces Structural Kolmogorov-Arnold Networks (KANs) that enhance convolutional neural networks by replacing fixed kernel weights with learnable functions, strategically positioned to optimize parameter efficiency. The authors propose three realizations鈥擲V-KAN, AG-KAN, and RF-KAN鈥攅ach varying in how the learnable functions interact with pixel values and filter shapes. Notably, RF-KAN achieves state-of-the-art performance on CIFAR-10 and CIFAR-100 with significantly fewer parameters than traditional per-edge KANs, demonstrating that the learned filter shape is crucial for maintaining high accuracy.
Learnable filter shapes in RF-KAN yield a performance boost that outstrips conventional convolutional approaches while slashing parameter counts by over 80%.
Convolutional Kolmogorov--Arnold Networks (KANs) replace the fixed weights of a convolutional kernel with learnable univariate functions. The dominant formulation attaches one such function to every kernel entry and lets it act on pixel values, expressive but parameter-heavy and prone to overfitting. We argue that the learnable functions are better placed in the \emph{structure} of the convolution than on each edge, and we organise the design space along a single axis: whether the function acts on the pixel \emph{values} or on the filter \emph{shape}. We study three realisations. SV-KAN applies one shared univariate function to the values and leaves the spatial filter free and static, aa classical convolution with a single learnable shared activation. AG-KAN keeps the shared value function but supplies the spatial structure through a content-adaptive Gaussian gate. RF-KAN instead moves the learnable functions onto the filter shape, building each filter from oriented ridge profiles expanded in a localised oscillatory (Morlet) wavelet basis with content-adaptive amplitudes. Under a matched four-layer protocol with in-run references and three seeds, RF-KAN and SV-KAN reach $88.47\pm0.10\%$ and $88.20\pm0.31\%$ on CIFAR-10 and $64.40\pm0.19\%$ and $64.57\pm0.30\%$ on CIFAR-100, at about $0.4$M parameters. At this matched scale the shape model and the simplest value model meet at the top, both above a plain convolution and every per-edge KAN we tested, including the official Gram variant, at roughly a fifth of the parameters. A controlled study attributes the RF-KAN gain to an intrinsically localised oscillatory basis and to content adaptivity, and an ablation that removes the learned shape entirely, leaving only the shared value function, collapses accuracy by over forty points, identifying the learned shape as the load-bearing ingredient at this scale.
