Search papers, labs, and topics across Lattice.
This paper introduces PIT-SUN, a novel empirical marginal recovery framework designed to address the challenges of estimating original-space conditional expectations in recommender systems, particularly under conditions of heavy-tailed, zero-inflated, and multimodal targets. By employing a bounded normal-score coordinate and variance-controlled recovery base, PIT-SUN effectively maintains expectation consistency while improving point accuracy and ranking quality. Experimental results across various datasets demonstrate significant enhancements in performance with minimal deployment overhead, making it a practical solution for real-world applications.
Expectation-consistency in recommender systems can be achieved without sacrificing performance, thanks to the PIT-SUN framework's innovative approach to empirical marginal recovery.
Estimating original-space conditional expectations is central to value-driven recommender systems, including dwell time, GMV, and LTV forecasting. Standard MSE is expectation-consistent in principle, but its gradients become unstable on heavy-tailed, zero-inflated, and multimodal targets, causing mean collapse and tail shrinkage. Target transformation alleviates this scale conflict, yet any useful nonlinear marginal transform loses expectation consistency under direct inversion. This is not an implementation oversight: a direct inverse-transform estimator is universally expectation-consistent only when the inverse transform is affine, which cannot simultaneously provide bounded tail compression. Existing conditionally linear recovery methods restore expectation consistency, but still leave open which coordinate, inverse lookup, recovery base, and deployment monitor should be selected for sparse complex marginals. We propose \textbf{P}robability-\textbf{I}ntegral-\textbf{TranS}formed \textbf{Un}biased recovery (\textbf{PIT-SUN}), a deployable empirical marginal recovery framework. PIT-SUN uses one empirical marginal table to define a bounded normal-score coordinate, its inverse-quantile lookup, a variance-controlled recovery base, and drift monitoring, then applies multiplicative SUN recovery to estimate the original-space expectation instead of directly inverting transformed predictions. Experiments on synthetic distributions, public benchmarks, large-scale industrial datasets, and online deployment show robust improvements in point accuracy, calibration, and ranking quality with lightweight deployment overhead.
