Lukasz Heldt

TokenMinds reveals that combining discrete SID-based user tokens with dense embeddings can significantly enhance user modeling in recommender systems at scale.

Transforming traditional signals into "soft tokens" could revolutionize how we integrate diverse data into Large Recommendation Models without compromising performance.

Forget slow prefix trees: STATIC unlocks massive speedups (up to 1033x) for constrained LLM decoding on GPUs/TPUs by vectorizing trie traversals into sparse matrix operations.