Search papers, labs, and topics across Lattice.
This study evaluates the effectiveness of large language models (LLMs) in providing feature-based rationales for next-basket repurchase recommendations, moving beyond traditional ranking accuracy. By analyzing two public grocery datasets and one proprietary retail dataset, the authors find that while LLMs do not outperform supervised rankers in ranking tasks, they can enhance the interpretability of recommendations through human-readable rationales. The results indicate that LLMs should be viewed as supplementary tools for generating explanations rather than as primary ranking systems, with the quality of rationales needing independent assessment.
LLMs may not lead in ranking accuracy for repurchase recommendations, but they excel at providing interpretable rationales that enhance user understanding.
Next-basket repurchase recommendation is commonly formulated as a ranking task: given a customer's purchase history, the system ranks previously purchased items that may be needed again. In production settings, however, ranking accuracy is only one component of recommendation quality. Customers may also benefit from concise evidence about why an item is recommended now. Large language models (LLMs) offer a potential way to surface such evidence through feature-based, human-readable rationales grounded in interpretable behavioral signals. We construct repurchase features spanning cadence, frequency, recency, user behavior, and item popularity, and evaluate LLMs on two public grocery datasets and one proprietary retail dataset. We investigate (1) whether off-the-shelf LLMs can use these features as next-basket scorers relative to heuristic and supervised rankers, and (2) whether LLM-cited features carry outcome-grounded ranking signal. For the latter, we compare LLM-cited features with model-specific attribution methods under a cross-model feature-masking protocol that measures ranking degradation after masking selected features. Our results show that LLM scores are not competitive with supervised rankers, suggesting that off-the-shelf LLMs should not be used as standalone repurchase recommenders. However, changes in prompt and evidence representation can improve outcome-grounded feature-masking results in some settings even when ranking performance does not improve; the effect is dataset-dependent and does not consistently match attribution baselines. These findings suggest a practical role for LLMs as validated explanation components rather than primary rankers, with rationale quality evaluated separately from ranking accuracy.
