Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
The authors resolve the theoretical limits of neural network compactness by determining the exact minimum hidden neuron count and parameter bit complexity required for arbitrary-accuracy uniform approximation of multivariate Hölder continuous functions. They prove that $d+1$ hidden neurons across two layers (widths $d$ and $1$) are strictly necessary and sufficient for standard feedforward networks with locally integrable activations, while also providing a practical construction requiring $d+2$ neurons using an elementary floor-exponential activation. This closes a foundational problem in approximation theory by showing that minimal-width networks can simultaneously saturate optimal architectural compactness and near-optimal $O(\varepsilon^{-d/\alpha}\log(1/\varepsilon))$ metric-entropy bit complexity.
Standard feedforward networks require just $d+1$ total hidden neurons to approximate $d$-dimensional continuous functions to arbitrary precision, settling the exact theoretical minimum while maintaining near-optimal bit complexity.
We study the minimum number of hidden neurons required for arbitrary-accuracy approximation of multivariate Hölder-continuous functions on $[0,1]^d$ and the associated encoding complexity. For $d\geq 2$, we construct a fixed, explicitly defined activation function for which a closed-form network with two hidden layers of widths $d$ and $1$ achieves arbitrary accuracy in the uniform norm. We prove that $d+1$ is the exact minimum total number of hidden neurons among standard feedforward networks with locally integrable activations and affine outputs. We further give a simpler construction using a single elementary activation that combines the floor and exponential functions. This construction requires three hidden layers of widths $d$, $1$, and $2$, only two neurons above the minimum. If a skip connection is allowed, widths $d$, $1$, and $1$ suffice. These constructions use explicit grid addressing and integer encoding of quantized function values. For a bounded $α$-Hölder class, they require $O(\varepsilon^{-d/α}\log(1/\varepsilon))$ bits, matching the metric-entropy lower bound up to a logarithmic factor.