Search papers, labs, and topics across Lattice.
This paper investigates the implications of dependent covariates and degenerate covariance matrices in over-parameterized linear regression, challenging the traditional assumption of independence. By deriving deterministic equivalents for prediction risk in a vanishing-ridge regime, the authors reveal that such degeneracies can lead to multiple descent phenomena, identifying specific configurations where variance peaks occur. The approach leverages a novel graph representation of the variance profile, utilizing maximum matchings and the Dulmage鈥揗endelsohn decomposition to pinpoint singular variance configurations.
Degenerate covariance structures can trigger multiple descent in regression, revealing unexpected peaks in prediction risk that challenge conventional wisdom.
Over-parameterized linear regression has been widely studied over the last decade. However, most existing works assume that the covariates are independent and that their covariance matrices are non-degenerate. In this paper, we relax both assumptions and derive deterministic equivalents for the prediction risk in a vanishing-ridge regime. We show that degeneracy of the covariance matrices and dependence can lead to multiple descent, and characterize where the corresponding peaks can occur. Our proofs use a novel graph representation of the variance profile. We show that maximum matchings and the Dulmage--Mendelsohn decomposition of the associated bipartite graph identify the configurations at which the variance becomes singular.
