Search papers, labs, and topics across Lattice.
This paper explores the deployment of Spectral Higher-Order Neural Networks (SHONNs), which utilize a novel parameterization leveraging spectral attributes to reduce the computational cost associated with neural hypergraphs. By implementing a weight-sharing scheme, the authors demonstrate significant improvements in performance and interpretability on N-bit parity tasks, a challenging benchmark in machine learning. The findings reveal that SHONNs offer a highly tunable hypothesis space, enhancing their expressivity while mitigating the parameter explosion typically associated with hypergraph models.
Spectral Higher-Order Neural Networks can drastically reduce computational costs while improving performance on notoriously difficult tasks like N-bit parity.
Neural hypergraphs are a natural generalization of neural networks, the reference models in modern machine learning. Yet, their deployment has proven demanding: the number of weighted hyperedges required leads to an intractable parameter explosion. However, a novel parametrization that leverages spectral attributes for neural hypergraphs has been recently proposed, that enables to recycle parameters via a weight sharing scheme and consequently yields a significant reduction of the associated computational cost. Preliminary tests carried out on spectral higher-order architectures pointed to meaningful improvements in both performance and interpretability. Building on these results, we advance the benchmarking efforts by evaluating the spectral higher order framework on N-bit parity tasks, a well-established testbed known to be particularly challenging. As we will convincingly argue, Spectral Higher-Order Neural Networks (SHONNs) possess a versatile and highly tunable hypothesis space.
