Search papers, labs, and topics across Lattice.
This paper introduces a novel framework for learning systems that utilizes Gibbs measures on hierarchical structures, shifting from traditional empirical risk minimization to an energy-based model defined by an interaction potential. By deriving nonlinear integral fixed-point equations, the authors establish a rigorous connection between empirical loss landscapes and probabilistic inference on trees, revealing the existence of multiple equilibrium learning states. The findings highlight that hierarchical learning systems can exhibit phase transitions, leading to diverse prediction regimes influenced by data-dependent kernels, thus expanding the understanding of energy-based learning dynamics.
Hierarchical learning systems can exhibit phase transitions, revealing multiple equilibrium states that challenge traditional views on model optimization.
We introduce a data-driven probabilistic framework for learning systems based on Gibbs measures on hierarchical structures. Unlike standard empirical risk minimization, where a dataset is used to identify a single optimal parameter, our approach transforms the empirical loss function into an interaction potential defining an energy-based model. The resulting Gibbs distribution describes a family of equilibrium learning states generated by the data. We formulate the consistency conditions of the associated finite-volume distributions and derive nonlinear integral fixed-point equations whose solutions characterize the admissible learning states. These equations provide a rigorous connection between empirical loss landscapes and probabilistic inference on trees. For translation-invariant solutions, the problem reduces to the analysis of positive compact operators induced by data-dependent kernels, allowing us to establish existence and uniqueness conditions in the one-dimensional setting. Furthermore, we show that hierarchical learning systems may exhibit phase-transition phenomena: for certain empirical kernels on Cayley trees, multiple Gibbs measures emerge beyond a critical inverse temperature, corresponding to distinct equilibrium prediction regimes. Numerical experiments with non-separable kernels illustrate the appearance of multiple solution branches and demonstrate the coexistence of several data-induced learning states. Our results provide a new perspective on energy-based learning, where data do not merely determine an optimal model through minimization but define an entire probabilistic landscape of possible inference states.
