Search papers, labs, and topics across Lattice.
This study systematically evaluated various parallel ensemble methods across 56 tabular classification tasks, leading to the formulation of best practice recommendations for their application. The recommendations were validated on an additional 28 tasks, demonstrating significant performance improvements over traditional single best models and matching or surpassing individual ensemble methods. Notably, the findings revealed that while Blending and Stacking exhibit inconsistent performance, Robust Soft Voting outperforms other methods, particularly in multiclass scenarios.
Robust Soft Voting outshines traditional methods, proving to be a game-changer in multiclass classification tasks.
Parallel ensemble methods were compared on $56$ small-to-medium tabular classification tasks drawn from OpenML CC18. A set of ``best practice'' recommendations on the use of ensemble methods was derived from these observations. It was later validated on 28 additional tasks using TabArena's precomputed data, where the recommendation set significantly outperformed Single Best and matched or exceeded individual ensemble methods. Two key observations were made. First, Blending and Stacking are inconsistent, but their inconsistencies are independent and happen on different tasks. Second, while Hard Voting's probabilistic classification is rather weak, a consequence of using vote proportions as posterior estimates, Robust Soft Voting's probabilistic classification is particularly successful, especially in the multiclass case.
