Search papers, labs, and topics across Lattice.
This paper explores the use of lattice rules, a type of quasi-Monte Carlo method, as training points for Deep Neural Networks (DNNs) with smooth activation functions. The authors derive explicit regularity bounds for these DNNs and demonstrate that by tailoring the training points and regularization to match the target function's regularity, they can achieve improved generalization error bounds independent of input dimension. Empirically, DNNs trained with this tailored lattice-based regularization outperform those using standard L2 regularization.
Lattice rules, a simple quasi-Monte Carlo technique, can significantly boost DNN generalization by providing tailored training points and regularization that exploits target function regularity.
This survey article is concerned with the application of lattice rules to Deep Neural Networks (DNNs), lattice rules being a family of quasi-Monte Carlo methods. They have demonstrated effectiveness in various contexts for high-dimensional integration and function approximation. They are extremely easy to implement thanks to their very simple formulation -- all that is required is a good integer generating vector of length matching the dimensionality of the problem. In recent years there has been a burst of research activities on the application and theory of DNNs. We review our recent article on using lattice rules as training points for DNNs with a smooth activation function, where we obtained explicit regularity bounds of the DNNs. By imposing restrictions on the network parameters to match the regularity features of the target function, we prove that DNNs with tailored lattice training points can achieve good theoretical generalization error bounds, with implied constants independent of the input dimension. We also demonstrate numerically that DNNs trained with our tailored regularization perform significantly better than with standard $\ell_2$ regularization.
