Search papers, labs, and topics across Lattice.
This paper refines Cover's function-counting theory to better understand binary classification in low-dimensional data settings by addressing the limitations of the general position assumption. By incorporating the low-dimensional structure of the data, the authors derive new dichotomy counts and extend Cover's separation capacity, allowing for a more nuanced analysis of classification capabilities. The findings reveal that acknowledging the underlying data structure significantly enhances our understanding of generalization in learning models.
Refining Cover's theory reveals that low-dimensional data structures can dramatically enhance classification capabilities, challenging traditional assumptions in machine learning.
The success of deep learning models in classification and regression is widely attributed to the low-dimensional structure that real-world data tend to exhibit, despite their high-dimensional representation. This work attempts to provide a mathematical framework for binary classification on low-dimensional data, building on Cover's (1965) function-counting theory. With our framework, we aim to address the question of how the low-dimensional structure of the data affects the classification capabilities of learning models. Cover's theory relies on a general position assumption that blinds it to the underlying data structure. We refine this assumption to account for the low-dimensionality of the data and derive dichotomy counts that reflect the data structure. We further extend Cover's separation capacity and problem of generalization to the low-dimensional setting, enabling the impact of the underlying data structure on both to be analyzed.
ETH