Search papers, labs, and topics across Lattice.
The paper introduces Gaussian Process Activations (GAPA), a post-hoc uncertainty quantification method that replaces standard nonlinearities in pretrained networks with Gaussian process activations. GAPA preserves the original model's point predictions by ensuring the posterior mean of the GP activations matches the original activations, while also providing closed-form epistemic variances in activation space. By using a sparse variational inducing-point approximation and k-NN subset conditioning, GAPA scales to large architectures and achieves competitive calibration and OOD detection performance compared to existing post-hoc methods, without requiring retraining or sampling.
Get calibrated uncertainty from your frozen pretrained models without retraining or Monte Carlo sampling: just swap in Gaussian Process Activations (GAPA).
Reliable uncertainty estimates are crucial for deploying pretrained models; yet, many strong methods for quantifying uncertainty require retraining, Monte Carlo sampling, or expensive second-order computations and may alter a frozen backbone's predictions. To address this, we introduce Gaussian Process Activations (GAPA), a post-hoc method that shifts Bayesian modeling from weights to activations. GAPA replaces standard nonlinearities with Gaussian-process activations whose posterior mean exactly matches the original activation, preserving the backbone's point predictions by construction while providing closed-form epistemic variances in activation space. To scale to modern architectures, we use a sparse variational inducing-point approximation over cached training activations, combined with local k-nearest-neighbor subset conditioning, enabling deterministic single-pass uncertainty propagation without sampling, backpropagation, or second-order information. Across regression, classification, image segmentation, and language modeling, GAPA matches or outperforms strong post-hoc baselines in calibration and out-of-distribution detection while remaining efficient at test time.
