Search papers, labs, and topics across Lattice.
This paper establishes non-asymptotic concentration inequalities for error probabilities in the Empirical Risk Principle (ERP), yielding both lower and upper bounds for minimal risk with high confidence. By relaxing the boundedness condition of the empirical risk function to Gaussian or exponential integrability, the authors demonstrate that the confidence in the lower bound is independent of the number of training parameters and input dimensions. The upper bound's confidence is shown to be robust when the sample size significantly exceeds the box dimension of the parameter set, enhancing the efficiency of detecting deficiencies in learning machines.
Lower bounds for minimal risk can be confidently established without dependence on model complexity, revolutionizing how we assess learning machine performance.
In this paper we prove some concentration inequalities for two types of error probabilities in the Empirical Risk Principle (ERP) in statistical learning, which provide a lower bound and an upper bound for the minimal risk (in terms of the minimal empirical risk) with non-asymptotic high confidence. The usual boundedness condition of the empirical risk function is relaxed to the Gaussian or exponential integrability condition. The confidence of the lower bound of the minimal risk is shown to be independent of the number of training parameters and the dimension of the input vectors, allowing one to detect the deficiency of a learning machine efficiently; and the confidence of the upper bound of the minimal risk is proved to be high provided that the sample size $n$ is much greater than the box dimension of the parameter set $螛$ in the Orlicz metric $d_{蠄_1}$ associated with the risk functions. Our work is based on Talagrand's concentration inequalities (the sharp versions by Bousquet and Klein-Rio), transport-entropy inequalities and the recent progress in the theory of empirical processes and statistical learning.
