Search papers, labs, and topics across Lattice.
This paper introduces Rashomon Alignment (RA), a novel measure for assessing functional similarity between models by employing a geometrical perspective rather than relying solely on distributional outputs. By analyzing over 90 datasets, the authors demonstrate that RA captures model alignment in a way that is independent of specific data distributions, revealing critical divergences between model alignment and predictive accuracy. The findings suggest that geometrical and distributional alignment offer complementary insights, enhancing model selection, ensemble construction, and interpretability in machine learning.
Geometrical model alignment reveals hidden divergences in predictive accuracy, challenging traditional measures of functional similarity.
We propose Rashomon Alignment (RA), a new measure to assess functional similarity between two models. Existing functional similarity measures are distributional, quantifying differences between outputs of models applied to real-world data. However, these measures can be regarded as ecologically valid only for regions in the input space represented by the available data. We introduce a geometrical perspective on functional model similarity, which estimates it across the entire data space, offering a comprehensive view of decision boundary alignment independent of any specific data distribution. We also propose geometric Rashomon Alignment as a measure of geometrical similarity, which is computed using data uniformly sampled from the instance space. We perform an experimental analysis on more than 90 datasets, examining critical cases where model alignment diverges from predictive accuracy. Our results show that geometrical and distributional alignment provide different and complementary perspectives on the similarity between models and algorithms. RA can be used for multiple purposes, including model selection, ensemble construction, and enhanced interpretability of machine learning models and algorithms.
