Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces ERBench, a comprehensive evaluation framework for symbolic regression algorithms focused on equation discovery from data. It addresses the limitations of existing benchmarks by emphasizing the recovery of known groundtruth formulas and assessing algorithm robustness across varying dimensionality, sampling sizes, distributions, and domains. The key finding is that ERBench provides a more reliable measure of an algorithm's potential for discovering unknown equations, which is crucial for practical applications in modeling natural phenomena.
ERBench reveals that traditional benchmarks may mislead researchers about an algorithm's true capabilities in equation discovery, emphasizing the need for robust evaluation metrics.
Equation discovery aims to automate the discovery of scientific models in the form of mathematical equations from data. Technically, equation discovery is implemented by symbolic regression algorithms. Performance of symbolic regression for equation discovery is measured along two dimensions: Prediction accuracy on test data, and recovery of known groundtruth formulas. For standard regression, accuracy is typically measured on in-domain test data, for instance, by splitting a data set randomly into training and test data. While this makes sense for in-domain interpolation, which is the common goal in ordinary regression, it can be a misleading proxy for true model discovery and generalization. The obvious alternative is to measure out-of-domain accuracy. However, obtaining challenging out-of-domain test data is a non-trivial problem. Therefore, we focus on equation recovery for evaluating symbolic regression algorithms for equation discovery. The rationale is that symbolic regression algorithms that perform well in recovering known groundtruth formulas are good candidates to perform well in unknown equation discovery. Existing benchmarks for symbolic regression include equation recovery tasks, however, with only a small number of groundtruth formulas that are publicly known. Moreover, these benchmarks place less emphasis on evaluating the robustness of algorithms in terms of their behavior under changing dimensionality, sampling size, sampling distribution and sampling domain. This, however, is of central importance to practitioners wanting to discover equations for modeling natural phenomena, since data is almost certainly noisy and comes from diverse domains, distributions, and sample sizes. To fill this gap, we introduce the Equation Recovery Benchmark (ERBench), a new evaluation framework designed to rigorously assess algorithms explicitly targeting the task of equation discovery.