Search papers, labs, and topics across Lattice.
The paper introduces Spectral Higher-Order Neural Networks (SHONNs), a novel approach to incorporate higher-order interactions within general-purpose feedforward neural networks, addressing the limitations of existing higher-order networks that primarily function as augmented GNNs. SHONNs reformulates the model using spectral attributes to overcome stability and parameter scaling challenges associated with weighted, higher-order forward propagations. The proposed method broadens the applicability of higher-order interactions beyond hypergraph-structured data.
Higher-order neural networks don't need hypergraphs: SHONNs unlock their power for general-purpose feedforward architectures by sidestepping stability and scaling issues.
Neural networks are fundamental tools of modern machine learning. The standard paradigm assumes binary interactions (across feedforward linear passes) between inter-tangled units, organized in sequential layers. Generalized architectures have been also designed that move beyond pairwise interactions, so as to account for higher-order couplings among computing neurons. Higher-order networks are however usually deployed as augmented graph neural networks (GNNs), and, as such, prove solely advantageous in contexts where the input exhibits an explicit hypergraph structure. Here, we present Spectral Higher-Order Neural Networks (SHONNs), a new algorithmic strategy to incorporate higher-order interactions in general-purpose, feedforward, network structures. SHONNs leverages a reformulation of the model in terms of spectral attributes. This allows to mitigate the common stability and parameter scaling problems that come along weighted, higher-order, forward propagations.
