Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
RecNextEval is introduced as a reference implementation for next-batch recommendation evaluation, addressing concerns about data leakage in existing RecSys evaluation pipelines. It enforces temporal validity by using a time-window data split, ensuring models are evaluated along a global timeline. The open-source library and GUI aim to promote more realistic and reproducible RecSys evaluation.
Data leakage might be silently inflating your RecSys metrics; RecNextEval offers a time-aware evaluation framework to expose it.
A good number of toolkits have been developed in Recommender Systems (RecSys) research to promote fair evaluation and reproducibility. However, recent critical examinations of RecSys evaluation protocols have raised concerns regarding the validity of existing evaluation pipelines. In this demonstration, we present RecNextEval, a reference implementation of an evaluation framework specifically designed for next-batch recommendation. RecNextEval utilizes a time-window data split to ensure models are evaluated along a global timeline, effectively minimizing data leakage. Our implementation highlights the inherent complexities of RecSys evaluation and encourages a shift toward model development that more accurately simulates production environments. The RecNextEval library and its accompanying GUI interface are open-source and publicly accessible.