Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
To overcome diminishing returns from simply expanding feature interaction capacity in CTR cross networks, the authors formalize "estimator scaling"—harnessing non-shared predictive variation across diverse estimators—and instantiate it via the RECursive Averaged Predictor (RECAP). Rather than incurring the prohibitive deployment footprint of traditional ensembles, RECAP extracts multi-estimator diversity through a parameter-efficient framework spanning cross-model distillation, training trajectory exponential moving averaging (EMA), and multi-route aggregation within a weight-shared recursive backbone. The approach establishes new state-of-the-art predictive performance across standard benchmarks while operating on a strictly superior performance-parameter Pareto frontier.
Pumping capacity into feature cross-networks yields diminishing returns, but harvesting non-shared estimator variance via recursive weight-sharing unlocks state-of-the-art CTR accuracy without standard ensemble deployment overhead.
Click-Through Rate prediction, a core task in recommendation and advertising systems, relies on modeling interactions among sparse categorical features. Explicit cross networks are a central paradigm for CTR prediction, and recent progress has largely come from increasing the interaction capacity of a single predictor through deeper cross networks and more expressive cross operators. We revisit whether continually increasing interaction capacity remains the most effective way to improve predictive performance, and find that its benefits quickly exhibit diminishing returns even as capacity continues to grow. This motivates a complementary scaling direction that we call estimator scaling, where additional resources are used to incorporate multiple related estimators rather than only enlarging a single predictor. Through theoretical analysis, we show that the gains from estimator scaling are governed by the amount of non-shared predictive variation available across estimators. However, exploiting this variation naively can be expensive: independently trained models provide substantial estimator diversity but require deployment cost to grow with ensemble size. This motivates a parameter-efficient realization of estimator scaling that can incorporate diversity from multiple estimator sources without maintaining multiple full models. Building on this view, we introduce RECursive Averaged Predictor (RECAP), a parameter-efficient recursive CTR model that operationalizes estimator scaling at three levels: distillation across independently trained models, exponential moving averaging over training trajectories, and aggregation over inference-time routes within a weight-shared recursive backbone. Experiments across multiple benchmarks establish new state-of-the-art predictive performance on standard benchmarks, while placing the RECAP on a favorable performance-parameter Pareto frontier.
Google Research