Search papers, labs, and topics across Lattice.
This paper reframes the oversmoothing problem in deep GNNs as a bifurcation phenomenon, where node features converge to a homogeneous fixed point. The authors theoretically prove that replacing standard monotone activations like ReLU with a specific class of functions can induce a bifurcation, destabilizing the homogeneous state and creating stable, non-homogeneous patterns that resist oversmoothing. They validate a predicted scaling law for the amplitude of these patterns and demonstrate the practical utility of a bifurcation-aware initialization on benchmark datasets.
Ditching ReLU can provably prevent oversmoothing in deep GNNs by inducing bifurcations that create stable, heterogeneous node representations.
Graph Neural Networks (GNNs) learn node representations through iterative network-based message-passing. While powerful, deep GNNs suffer from oversmoothing, where node features converge to a homogeneous, non-informative state. We re-frame this problem of representational collapse from a \emph{bifurcation theory} perspective, characterizing oversmoothing as convergence to a stable ``homogeneous fixed point.'' Our central contribution is the theoretical discovery that this undesired stability can be broken by replacing standard monotone activations (e.g., ReLU) with a class of functions. Using Lyapunov-Schmidt reduction, we analytically prove that this substitution induces a bifurcation that destabilizes the homogeneous state and creates a new pair of stable, non-homogeneous \emph{patterns} that provably resist oversmoothing. Our theory predicts a precise, nontrivial scaling law for the amplitude of these emergent patterns, which we quantitatively validate in experiments. Finally, we demonstrate the practical utility of our theory by deriving a closed-form, bifurcation-aware initialization and showing its utility in real benchmark experiments.
