Search papers, labs, and topics across Lattice.
This paper introduces Variational Kernel Design (VKD), a framework for specifying internal noise mechanisms in deep networks based on learning desiderata. VKD derives a novel noise mechanism called Gaussian Chaos Noise (GCh) from a quadratic maximum-entropy principle, resulting in a Dirichlet Green kernel geometry. GCh demonstrates improved calibration and robustness to distribution shift on ImageNet and ImageNet-C compared to hard binary masks, while maintaining competitive accuracy.
Forget dropout – Gaussian Chaos Noise offers provable control over representation deformation and boosts calibration in deep networks.
Internal noise in deep networks is usually inherited from heuristics such as dropout, hard masking, or additive perturbation. We ask two questions: what correlation geometry should internal noise have, and is the implemented perturbation compatible with the representations it acts on? We answer these questions through Variational Kernel Design (VKD), a framework in which a noise mechanism is specified by a law family, a correlation kernel, and an injection operator, and is derived from learning desiderata. In a solved spatial subfamily, a quadratic maximum-entropy principle over latent log-fields yields a Gaussian optimizer with precision given by the Dirichlet Laplacian, so the induced geometry is the Dirichlet Green kernel. Wick normalization then gives a canonical positive mean-one gate, Gaussian Chaos Noise (GCh). For the sample-wise gate used in practice, we prove exact Gaussian control of pairwise log-ratio deformation, margin-sensitive ranking stability, and an exact expected intrinsic roughness budget; hard binary masks instead induce singular or coherence-amplified distortions on positive coherent representations. On ImageNet and ImageNet-C, GCh consistently improves calibration and under shift also improves NLL at competitive accuracy.
