Search papers, labs, and topics across Lattice.
This paper investigates the effectiveness of deep ensembles for uncertainty quantification in message-passing GNNs across seven datasets. It finds that ensembles offer minimal improvement over single models, primarily due to epistemic collapse, where independently trained networks converge to similar predictions. The study attributes this collapse to functional convexity, where different parameter solutions lead to nearly identical behavior, thus limiting the ensemble's ability to capture epistemic uncertainty.
Deep ensembles, the go-to for uncertainty in standard deep learning, largely fail for graph neural networks due to a surprising "epistemic collapse" where individual models make nearly identical predictions.
While deep ensembles are widely considered to be the default method for uncertainty quantification in deep learning, their effectiveness for graph-structured data is often simply assumed based on successes in domains like computer vision. We investigate standard deep ensembles specifically for message-passing graph neural networks. Benchmarking across seven datasets representing varied tasks and complexities, we reveal that ensembles provide surprisingly little improvement over a single model. Instead, the observed marginal gains stem primarily from stabilizing optimization noise in point predictions rather than yielding meaningfully better uncertainty estimates. Through an aleatoric-epistemic decomposition, we identify epistemic collapse: independently trained networks consistently converge to overly similar predictions. Because disagreement is the fundamental mechanism through which ensembles capture epistemic uncertainty, this lack of diversity neutralizes their key advantage. Analyzing this phenomenon further, we suggest this collapse is driven by functional rather than weight-space convexity, where distinct parameter solutions induce almost identical behavior. Our results suggest that deep ensemble success does not seamlessly transfer to graph machine learning.
