Search papers, labs, and topics across Lattice.
This paper introduces a novel synthetic data model based on critical mean-field percolation clusters to enhance the evaluation of interpretability methods in neural networks. By leveraging the hierarchical structure and power-law characteristics of percolation data, the authors create a scalable and analytically tractable framework that allows for the generation of complex datasets without hyperparameter tuning. The key finding reveals that latent variables derived from this model can be effectively decoded from neural network activations, establishing a new benchmark for interpretability research.
Ground-truth latent variables from a new synthetic data model can be decoded from neural network activations, revealing insights into interpretability like never before.
Neural networks learn features that reflect the hierarchical, multi-scale structure of natural data. Synthetic datasets used to evaluate interpretability methods typically lack this structure, limiting their value as realistic toy models. To close this gap, we introduce a family of synthetic datasets consisting of hierarchical functions defined on critical mean-field percolation clusters embedded in a high-dimensional data space. The percolation data consists of sparse, low-dimensional fractal clusters with a power-law size distribution. Latent variables modeling a taxonomic hierarchy generate each data point's target value. The data model is analytically tractable with known critical exponents that fix its properties without requiring hyperparameter tuning. We leverage a mapping between percolation clusters, random trees, and additive coalescence to propose an almost linear-time algorithm to jointly sample a random tree and its hierarchical latent decomposition, enabling data generation at arbitrary scale. Using probing experiments, we find that the model's ground-truth latent variables can be linearly decoded from neural network activations. Together, sparsity, self-similarity, power-law statistics, and analytical tractability make critical percolation a principled testbed for interpretability research.
