Search papers, labs, and topics across Lattice.
This paper introduces SIDScope, a diagnostic tool designed to evaluate the coherence and structure of Semantic-ID mappings between item tokenizers and generative recommenders. By analyzing nine source-traced tokenizer exports from various families, SIDScope uncovers that interface health is determined by multiple signals rather than a singular metric, revealing that prefix alignment is crucial for candidate exposure during retrieval. The findings highlight the complexities involved in maintaining effective mappings, particularly emphasizing that simply repairing mappings does not guarantee the restoration of generator functionality without additional checks.
Interface health in generative recommendation systems is multi-signal, revealing that prefix alignment is critical for candidate exposure but weakens under certain scoring conditions.
Semantic-ID mappings are reusable interfaces between item tokenizers and generative recommenders, yet released mappings rarely state whether they are coherent, what structure they expose, how generated paths resolve, or what must be revalidated after a refresh. SIDScope is a source-traced diagnostic resource for these decisions. It normalizes item-to-code artifacts, verifies provenance and joins, profiles mapping structure, compares paired revisions, and accounts for path-to-item outcomes in generated traces. Across nine source-traced tokenizer exports from seven families on Amazon and Yelp data - eight executable routes plus one auditable snapshot - SIDScope reveals that interface health is multi-signal rather than scalar. Its central finding is mechanism-conditional: prefix alignment strongly tracks held-out candidate exposure when retrieval consumes SID prefixes, then weakens as scoring becomes prefix-independent. Trained trace accounting exposes a second hidden gap: a valid target path can survive without uniquely retrieving the target item by 1.2-3.0 percentage points. A refresh case establishes a third: repairing the mapping does not by itself restore an inherited generator; model reuse requires a separate handoff check. The package provides frozen evidence summaries, conformance reports, trace labels, table builders, and CPU-only verifiers. It supports decisions about artifact readiness, interface risks, and revalidation before model reuse.
