Search papers, labs, and topics across Lattice.
This paper introduces Orthogonal Constrained Projection (OCP), a novel method to improve Item-Id embedding representation in industrial recommendation systems subjected to sparse scaling. OCP enforces orthogonality during backpropagation, aligning the singular value spectrum of learned embeddings with an orthogonal basis, thereby increasing singular entropy and preserving isotropic generalized features. Experiments on JD.com's recommendation system show that OCP accelerates loss convergence, enhances scalability, and achieves a 12.97% increase in UCXR and an 8.9% uplift in GMV.
Orthogonal constraints can rescue sparse embeddings in recommender systems from representation collapse, unlocking significant performance gains in large-scale industrial deployments.
In industrial commodity recommendation systems, the representation quality of Item-Id vocabularies directly impacts the scalability and generalization ability of recommendation models. A key challenge is that traditional Item-Id vocabularies, when subjected to sparse scaling, suffer from low-frequency information interference, which restricts their expressive power for massive item sets and leads to representation collapse. To address this issue, we propose an Orthogonal Constrained Projection method to optimize embedding representation. By enforcing orthogonality, the projection constrains the backpropagation manifold, aligning the singular value spectrum of the learned embeddings with the orthogonal basis. This alignment ensures high singular entropy, thereby preserving isotropic generalized features while suppressing spurious correlations and overfitting to rare items. Empirical results demonstrate that OCP accelerates loss convergence and enhances the model's scalability; notably, it enables consistent performance gains when scaling up dense layers. Large-scale industrial deployment on JD.com further confirms its efficacy, yielding a 12.97% increase in UCXR and an 8.9% uplift in GMV, highlighting its robust utility for scaling up both sparse vocabularies and dense architectures.
