Search papers, labs, and topics across Lattice.
This study explores the use of a Matryoshka Sparse Autoencoder (MSAE) to extract interpretable hierarchical structures from collaborative filtering embeddings in recommender systems. By applying this method to a large-scale matrix factorization model trained on the Amazon Fashion dataset, the authors demonstrate that the resulting latent features can achieve semantic coherence and disentanglement, addressing the interpretability issues of traditional sparse autoencoders. The intervention on gender-associated latent neurons further illustrates the potential for principled manipulation of recommendation behavior based on interpretable features.
Collaborative filtering embeddings reveal recoverable hierarchical structures, enabling targeted interventions in recommendation systems based on interpretable latent factors.
Latent factor models such as matrix factorization are widely used in recommender systems, yet the learned embedding dimensions typically lack explicit semantic interpretation. This opacity limits transparency, explainability, and principled intervention in recommendation behavior. While sparse autoencoders (SAEs) have recently been used to extract monosemantic features from dense neural representations, standard SAEs suffer from scaling pathologies including feature splitting, feature absorption, and feature composition, which degrade interpretability as dictionary size increases. In this work, we investigate whether hierarchical sparse representations can reveal interpretable structure in collaborative filtering embeddings. We train a large-scale matrix factorization recommender system on the Amazon Fashion dataset and apply a Matryoshka Sparse Autoencoder (MSAE) to the learned embeddings. We analyze the resulting latent features through metadata alignment and LLM-generated labeling to assess semantic coherence and disentanglement. Finally, we show an intervention on a subset of gender associated latent neurons that emerged from the analysis. Our findings suggest that collaborative filtering embeddings contain recoverable hierarchical structure, and that Matryoshka training provides a principled mechanism for exposing interpretable latent factors in interaction-driven recommendation models.
