Search papers, labs, and topics across Lattice.
This paper establishes a mathematical equivalence between hierarchical decision trees and diffusion processes by identifying a shared optimization principle called Global Trajectory Score Matching (GTSM). Gradient boosting is shown to be asymptotically optimal for GTSM in an idealized setting, bridging the gap between these seemingly distinct model classes. The authors leverage this unification to develop TreeFlow, a fast and high-fidelity generative model for tabular data, and DSMTree, a distillation method that effectively transfers decision tree logic into neural networks.
Decision trees and diffusion models are secretly doing the same thing: optimizing a shared objective called Global Trajectory Score Matching.
Decision trees and diffusion models are ostensibly disparate model classes, one discrete and hierarchical, the other continuous and dynamic. This work unifies the two by establishing a crisp mathematical correspondence between hierarchical decision trees and diffusion processes in appropriate limiting regimes. Our unification reveals a shared optimization principle: \emph{Global Trajectory Score Matching (GTSM)}, for which gradient boosting (in an idealized version) is asymptotically optimal. We underscore the conceptual value of our work through two key practical instantiations: \treeflow, which achieves competitive generation quality on tabular data with higher fidelity and a 2\times computational speedup, and \dsmtree, a novel distillation method that transfers hierarchical decision logic into neural networks, matching teacher performance within 2\% on many benchmarks.
