Track new results across pure and applied mathematics.
Search papers, labs, and topics across Lattice.
This paper introduces FORTE, a model-agnostic framework designed to address the feature-lifecycle asymmetry in industrial recommender systems by enabling efficient discovery, validation, and deprecation of features. FORTE employs grouped ablation for interaction-aware evaluation and lineage tracing to identify downstream consumers, allowing for confident deprecation decisions that are predictive of online A/B test outcomes. Validation on two large-scale recommender systems demonstrated FORTE's ability to identify significant feature sets for deprecation without causing regressions, leading to substantial reductions in model complexity, improved serving throughput, and cost savings.
Industrial recommender systems can now deprecate nearly half their features without performance loss, unlocking significant efficiency gains.
Production recommender systems have feature-lifecycle asymmetry: adding new features must produce gains on some metric, while feature removal must not regress any metric. In practice, feature deprecation is hard to prioritize as a feature can fail to be removed if it interacts with other features, its importance changes daily, it cannot be evaluated before production models change or data drift occurs, it is in contention for A/B test space with other launches, it causes regressions in an online A/B test, or it is consumed by multiple models that need to coordinate its deprecation. We present FORTE, a model-agnostic production framework for discovering, validating, and removing features. Its efficient perturbation implementation allows daily feature evaluation. To account for interactions, it evaluates candidate feature sets constructed from all the low-rank features through grouped ablation and its offline results are sufficiently predictive of online A/B tests to allow deprecations to proceed to online testing with confidence. Lineage tracing identifies all downstream consumer models to realize the full savings of removing a feature from the training set. We validate FORTE over 2 large-scale recommender systems and find that it identifies deprecable feature sets which do not cause online regressions, while the impact on efficiency depends on model characteristics. On a search-relevance model, FORTE resulted in 3 nested candidate sets of 32, 48, and 59 features. Online guardrails rejected the largest treatment. The 48-feature set was selected for deprecation, removing 43.3% of the model's features. On a large ranking model, FORTE identified 8 candidates for removal. Retraining and A/B testing confirmed no regression. Its launch reduced the trainable parameters by 29.9%, improved serving throughput by 19.8-25.5%, and reduced annual training costs by hundreds of thousands of dollars.