Search papers, labs, and topics across Lattice.
This paper investigates the infinite-width Gaussian-process limit of random neural networks using tensor programs, establishing a quantitative convergence theory in Wasserstein distance. The authors derive explicit finite-width error bounds, demonstrating that the error between finite-network executions and their Gaussian-process limits scales inversely with the square root of the network widths. This architecture-agnostic framework encompasses various models, including feed-forward networks and weight-sharing schemes applicable to recurrent and transformer architectures.
Finite-width neural networks converge to their Gaussian-process limits with error bounds that shrink as the network width increases, revealing a surprising robustness across architectures.
We study the infinite-width Gaussian-process limit of random neural networks through the lens of tensor programs, and we provide a quantitative convergence theory in Wasserstein distance. Our main result gives explicit finite-width error bounds, of order inverse square-root of the widths between finite-network executions and their Gaussian-process limits. The framework is architecture-agnostic and covers feed-forward models together with weight-sharing schemes relevant for recurrent and transformer-type architectures.
