Search papers, labs, and topics across Lattice.
This paper introduces a unifying theoretical framework for unsupervised concept extraction, framing it as identifying a generative model. They provide a meta-theorem that reduces identifiability guarantee proofs to characterizing the intersection of two sets. The framework simplifies proving identifiability for existing methods like sparse autoencoders and transcoders, enabling the development of more principled concept extraction techniques.
Concept extraction's identifiability problem just got a lot easier, thanks to a new framework that turns guarantee proofs into set intersection problems.
Techniques for concept extraction, such as sparse autoencoders and transcoders, aim to extract high-level symbolic concepts from low-level nonsymbolic representations. When these extracted concepts are used for downstream tasks such as model steering and unlearning, it is essential to understand their guarantees, or lack thereof. In this work, we present a unified theoretical framework for unsupervised concept extraction, in which we frame the task of concept extraction as identifying a generative model. We present a general meta-theorem for identifiability, which reduces the problem of establishing identifiability guarantees to the problem of characterizing the intersection of two sets. As we demonstrate on a range of widely-used approaches, this meta-theorem substantially simplifies the task of proving such guarantees, thus paving the way for the development of new, principled approaches for concept extraction.
