Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces a comprehensive algorithmic framework for optimal decision trees (ODTs) that consolidates and evaluates 18 different search strategies for model optimization. By empirically analyzing these strategies, the authors reveal that the best-performing method significantly enhances anytime performance for classification tasks and drastically reduces runtime for regression tasks by over tenfold. This work clarifies the contributions of various search strategies, providing a unified perspective that can guide future research in scalable decision tree models.
The best search strategy for optimal decision trees boosts classification performance and slashes regression runtime by over 90%.
Optimal decision trees (ODTs) are compact, interpretable machine learning models that globally optimize a given objective, but their scalability remains challenging. While recent work has proposed a variety of search strategies to improve scalability, the precise contribution of each strategy remains unclear. To address this gap, we introduce a general algorithmic framework for ODTs that instantiates previously used search strategies and enables the definition of new ones. This provides a common lens through which to understand and compare different strategies, which we use to empirically investigate the effect of 18 search strategies. Compared to the state of the art, the best strategy in our evaluation achieves significantly better anytime performance for classification, and improves runtime by more than an order of magnitude for regression.
TU Delft