Track the frontier, from condensed matter to cosmology.
Search papers, labs, and topics across Lattice.
This paper investigates how neural networks reorganize their internal Boolean function representations during training and data accumulation, specifically in the context of parity learning. By analyzing the sign table of the trained model as a normalized state with an effective Schmidt rank, the authors observe a transition from broad differentiation with small data additions to target-selective recoherence with larger additions. This process reveals a distinct ordering of learning aspects: constraint identifiability, function-state organization, and predictive accuracy, with the learned state continuing to reorganize even after data constraints stabilize.
Neural network function states reorganize even after data constraints stabilize, revealing a distinct learning progression from identifiability to predictive accuracy.
We study how a neural network reorganizes its learned function as training proceeds and as previously unseen parity data accumulate. The trained model is itself a Boolean function on binary strings of finite length. We then evaluate it on all data, inside and outside the training set, and read off its complete sign table. Its $\pm1$ entries are the amplitudes of a normalized state, dubbed the function-state, whose inverse subsystem purity $K_f=1/\textrm{Tr}(ρ_A^2)$ is an effective Schmidt rank. We find that small data additions broaden the Schmidt spectrum, whereas larger additions reconcentrate it in the target mode. The former is called differentiation because the model distinguishes inputs more sharply. The latter is called target-selective recoherence, where the model settles on the single rule shared by the added data. As the dataset grows, the connectivity of the constraint graph defined by the training set increases monotonically, whereas the learned inverse purity changes nonmonotonically. On a fixed connected dataset, as learning proceeds, the Schmidt spectrum begins to reorganize, followed by gains in target alignment and later by improvement in predictive accuracy. This ordering distinguishes three aspects of learning: constraint identifiability, function-state organization, and predictive accuracy. At fixed data size, a function that has already been learned can re-differentiate, an event we call retreat, without changing graph connectivity. Recovery reconcentrates the Schmidt spectrum in a similar target-selective pattern, without exact recurrence of the Boolean state. Thus the percolation of data constraints fixes the identifiable support, while the reduced state produced by training continues to reorganize even after that support stops changing.