Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper reinterprets the Forward-Forward (FF) algorithm through the lens of likelihood-ratio testing, establishing that the scalar goodness metric—defined as the sum of squared activations—serves as a sufficient statistic for distinguishing between zero-mean populations with varying scales. By generalizing this concept to anisotropic and heavy-tailed distributions, the authors derive a Mahalanobis goodness and a saturating statistic that highlight the importance of divisive normalization in preserving evidence during aggregation. The findings also clarify the role of inter-layer normalization in maintaining energy while mitigating depth collapse, providing a deeper understanding of the FF algorithm's mechanisms.
The scalar goodness in Forward-Forward learning is not just a heuristic; it’s a critical statistic for likelihood-ratio testing that reveals deeper insights into model behavior across different population types.
The Forward-Forward (FF) algorithm trains each layer locally, so that a scalar goodness - the sum of squared activations - is high on real inputs and low on contrastive ones, with activations normalized between layers. Both choices are usually treated as heuristics. Under an explicit generative model they are not: the squared goodness is the sufficient statistic of a likelihood-ratio test between two zero-mean populations differing in scale, and the FF threshold is its boundary. It generalizes: anisotropic populations yield a Mahalanobis goodness, the plain square being its isotropic case; heavy-tailed populations yield a saturating statistic whose slope is a posterior precision - divisive normalization - with bounded evidence and an advantage only under aggregation. The same lens characterizes the inter-layer normalization: it must remove the length while preserving per-coordinate energy, explaining a depth collapse we observe under unit-norm normalization; and the pairwise objective admits a scale-inflation shortcut that a whitened goodness removes.