Search papers, labs, and topics across Lattice.
This paper introduces "diverse dictionary learning," a framework for recovering latent variables and their relationships from observational data when full identifiability is unattainable due to unknown generating processes. The key result is the provable recovery of set-theoretic relationships (intersections, complements, symmetric differences) between latent variables, even with arbitrary observation functions, by leveraging a simple inductive bias. This allows for the construction of structured views of the hidden world and, under sufficient diversity, full identifiability.
Even when you can't fully identify latent variables, provably recovering their set-theoretic relationships unlocks structured understanding of the hidden world.
Given only observational data $X = g(Z)$, where both the latent variables $Z$ and the generating process $g$ are unknown, recovering $Z$ is ill-posed without additional assumptions. Existing methods often assume linearity or rely on auxiliary supervision and functional constraints. However, such assumptions are rarely verifiable in practice, and most theoretical guarantees break down under even mild violations, leaving uncertainty about how to reliably understand the hidden world. To make identifiability actionable in the real-world scenarios, we take a complementary view: in the general settings where full identifiability is unattainable, what can still be recovered with guarantees, and what biases could be universally adopted? We introduce the problem of diverse dictionary learning to formalize this view. Specifically, we show that intersections, complements, and symmetric differences of latent variables linked to arbitrary observations, along with the latent-to-observed dependency structure, are still identifiable up to appropriate indeterminacies even without strong assumptions. These set-theoretic results can be composed using set algebra to construct structured and essential views of the hidden world, such as genus-differentia definitions. When sufficient structural diversity is present, they further imply full identifiability of all latent variables. Notably, all identifiability benefits follow from a simple inductive bias during estimation that can be readily integrated into most models. We validate the theory and demonstrate the benefits of the bias on both synthetic and real-world data.
