Search papers, labs, and topics across Lattice.
This paper introduces a novel method for measuring singular structures in trained neural networks without relying on alignment or descent conditions. By utilizing the directional-Fisher rate, the authors can determine the order of dead directions and classify them into genuine singularities or flat gauge symmetries, effectively recovering architecture-predicted orders across various network types. The findings reveal a deterministic approach to understanding network behavior, with implications for how singular structures influence data fluctuations within trained models.
Dead directions in neural networks can be classified and quantified without traditional alignment methods, revealing deeper insights into network architecture and behavior.
We give a descent-free, alignment-free measurement of singular structure on trained networks. At a single frozen checkpoint the read recovers the order $k$ of each dead direction from the directional-Fisher rate, the master invariant from which the per-direction learning coefficient $1/(2k)$ follows exactly, in whatever basis the optimizer left. The same read classifies each direction, separating a genuine singularity, whose order the architecture fixes, from a flat gauge symmetry; the directional-Fisher magnitude settles the cases the order cannot. A pluggable detector supplies the directions for transformer, convolutional, and normalisation layers. The read recovers the architecture-predicted order across constructed cells and trained networks, including a fine-tuned vision transformer whose dead structure is the LayerNorm-kernel gauge and a from-scratch one whose compressed MLP forms a node-death at its activation order. Where the singular structure enumerates, the per-direction orders assemble, through the typed intersection of the loci, into the global coefficient $(位, m)$ matching the closed form. The method removes the canonical-alignment and descent preconditions of the underlying rate result, turning order-recovery into a deterministic, architecture-general reading. We then map its reach into the Watanabe triple: the order determines the universal singular fluctuation $谓(k)$, though a trained network's realized $谓$ falls below it as the live structure absorbs the dead direction's data fluctuation, and the multiplicity recovers from the dominant structure under a single-locus assumption.
