Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
The paper introduces Simplex-Constrained Sparse Bagging (SCSB), a novel framework that enhances bootstrap-based bagging ensembles by optimizing the voting power of base estimators based on their local competence. By formulating ensemble pruning and calibration as a joint optimization problem over the probability simplex, SCSB effectively addresses the limitations of uniform priors and reduces model overconfidence. The method achieves up to 96% compression of ensembles while improving inference speed and probability calibration, all without sacrificing generalization accuracy.
Achieving up to 96% ensemble compression, SCSB transforms bagging by tailoring voting power to estimator competence, enhancing both speed and calibration.
We present Simplex-Constrained Sparse Bagging (SCSB), a mathematically rigorous framework for post-training compression and probability calibration of bootstrap-based bagging ensembles. Standard bagging ensembles (such as Random Forests, Bagged SVMs, and Bagged Neural Networks) assign uniform voting power to all constituent estimators. However, this naive uniform prior ignores the varying local competence of base estimators and contributes to model overconfidence. We formulate ensemble pruning and calibration as a joint optimization problem over the probability simplex by minimizing the Out-Of-Bag (OOB) loss. To induce sparsity, we address the theoretical"L1-simplex paradox"-- the mathematical reality that the L1 norm is constant on the simplex and fails to prune -- by introducing a concave quadratic penalty. SCSB is model-agnostic and achieves up to 96% ensemble compression, yielding linear inference speedups and superior probability calibration (lowered Expected Calibration Error) while preserving or enhancing generalization accuracy.