Search papers, labs, and topics across Lattice.
The authors establish a general theoretical framework to quantify how averaging-based ensembling strategies confer algorithmic stability against arbitrary data perturbations. They prove that the stability of an ensembled algorithm is fundamentally bounded by the operator norm of a covariance operator characterizing the ensembling process. This operator-theoretic formulation yields substantially sharper stability and generalization guarantees than traditional differential privacy reductions across diverse practical perturbation regimes.
Standard differential privacy reductions drastically underestimate how robust ensemble methods actually are, missing an exact connection between algorithmic stability and the spectral norm of the ensembling covariance operator.
Algorithmic stability refers to the property of an algorithm being insensitive to perturbations of the input data, where the type of perturbation may vary depending on the setting. In this work, we develop a general framework to quantify the extent to which any ensembling strategy defined via averaging can yield stability guarantees for any type of data perturbation. Our main theoretical result is a guarantee on the stability of this ensembled algorithm, given in terms of the norm of a certain covariance operator that describes the ensembling process. We show how our general framework yields interpretable and intuitive insights in several examples of perturbations of practical interest, and provides much sharper guarantees than those obtained from privacy considerations.
