Search papers, labs, and topics across Lattice.
This paper investigates neighbourhood selection in high-dimensional networks, revealing that while traditional linear models can effectively identify connections when the true model is linear, they falter under nonlinearity without appropriate penalties. The authors demonstrate that incorporating a ridge penalty significantly reduces mean squared error and test variance, leading to more accurate neighbourhood estimates. Notably, they connect their findings to the double descent phenomenon, suggesting that model space volume must be considered in penalties for effective neighbourhood selection in high-dimensional contexts.
Minimum description length outperforms traditional methods in high-dimensional neighbourhood selection, achieving lower false positive rates even when the true model is misspecified.
To avoid missing important variables and their connections in networks, more and more variables are included in network analysis. Here we show that in a setting with many more parameters than observations (high-dimensional) it is possible to get a conservative (i.e., low false positive rate) estimate of the neighbourhood for each node (which connections are in the network). A neighbourhood is often estimated with a linear model, and this leads to two interesting cases: (i) If the true model is linear, then neighbourhood selection work reasonably well, and (ii) if the true model is nonlinear, then neighbourhood selection requires a penalty for the high dimensions. Here we show the impact of the ridge parameter on the mean squared error, and how this leads to low test variance and hence to neighbourhoods with large numbers of edges. We connect these insights with results from machine learning, where the so-called double descent (when more parameters are included than observations, the mean squared error goes down a second time) has put the traditional view on model selection upside down. Essentially, for adequate neighbourhood selection in models with a large number of parameters, the volume of the model space needs to be included in the penalty. Most neighbourhood selection methods (e.g., Lasso, AIC, BIC) lead to spurious edges (high false positive rate), but we prove that in the high-dimensional setting, minimum description length leads to correct neighbourhood selection or smaller (low false positive rates) in both cases when either the model is correctly or incorrectly assumed linear
