Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper addresses the challenge of covariate shift in nonparametric regression by introducing a sparse-penalized deep neural network (SPDNN) estimator that adapts to discrepancies between source and target distributions. The authors establish non-asymptotic error bounds for both quantile and Huber regression, demonstrating that their SPDNN estimators achieve minimax optimal convergence rates even under dependent observations. This work leverages a two-step pre-training procedure to estimate the density ratio when it is unknown, significantly enhancing regression performance in practical applications.
SPDNN estimators can adaptively achieve optimal convergence rates under covariate shift, even when the density ratio is unknown.
Covariate shift often occurs because, in many real applications, the source and the target observations may be generated from different distributions. In this case, the standard metric under the source distribution is not appropriate. This paper considers deep neural network estimators for nonparametric quantile and Huber regression under covariate shift and from dependent observations. We deal with a generalized Bernstein-type inequality that is satisfied by many classical models, including i.i.d. observations, $φ$-mixing, strong mixing, and $\mathcal{C}$-mixing processes. To perform the covariate shift phenomenon, we propose a sparse-penalized deep neural network (SPDNN) estimator that takes into account the discrepancy between the source and target distributions of the data. When the density ratio (between the source and target distributions of the covariate) is unknown, a two steps pre-training procedure is carried out: the first step is devoted to the construction of a least squares SPDNN estimator of the density ratio; which is used in the second step to perform a pre-training reweighted SPDNN estimator of the regression function. For both the quantile and the Huber regression, non-asymptotic error bounds of the proposed SPDNN estimators are established in the class of Hölder smooth functions. These estimators can adaptively attain (up to a logarithmic factor) the minimax optimal convergence rate from i.i.d. data as well as from several classical time series models.