Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
ConceptTracer is introduced as an interactive tool for analyzing neural representations by quantifying concept saliency and selectivity using information-theoretic measures. It allows users to identify neurons that strongly respond to specific human-interpretable concepts within tabular foundation models like TabPFN. The tool's utility is demonstrated by discovering interpretable neurons within TabPFN, providing a framework for understanding concept encoding in neural networks.
Uncover the "why" behind neural network decisions with an interactive tool that reveals which neurons light up for specific concepts.
Neural networks deliver impressive predictive performance across a variety of tasks, but they are often opaque in their decision-making processes. Despite a growing interest in mechanistic interpretability, tools for systematically exploring the representations learned by neural networks in general, and tabular foundation models in particular, remain limited. In this work, we introduce ConceptTracer, an interactive application for analyzing neural representations through the lens of human-interpretable concepts. ConceptTracer integrates two information-theoretic measures that quantify concept saliency and selectivity, enabling researchers and practitioners to identify neurons that respond strongly to individual concepts. We demonstrate the utility of ConceptTracer on representations learned by TabPFN and show that our approach facilitates the discovery of interpretable neurons. Together, these capabilities provide a practical framework for investigating how neural networks like TabPFN encode concept-level information. ConceptTracer is available at https://github.com/ml-lab-htw/concept-tracer.