Search papers, labs, and topics across Lattice.
This paper tackles nonstationary classification by conditioning a classifier on a sequence of historical labeled examples retrieved via a learned discrete retrieval mechanism. The retrieval mechanism uses input-dependent queries, trained end-to-end with the classifier using a score-based gradient estimator, allowing the full corpus of historical data to remain on an arbitrary filesystem during training and deployment. Experiments on synthetic benchmarks and Amazon Reviews '23 demonstrate improved robustness to distribution shift compared to standard classifiers, with predictable VRAM scaling.
Forget retraining: this learned retrieval approach lets classifiers adapt to distribution shift by querying a vast historical dataset on demand.
Nonstationarity is ubiquitous in practical classification settings, leading deployed models to perform poorly even when they generalize well to holdout sets available at training time. We address this by reframing nonstationary classification as time series prediction: rather than predicting from the current input alone, we condition the classifier on a sequence of historical labeled examples that extends beyond the training cutoff. To scale to large sequences, we introduce a learned discrete retrieval mechanism that samples relevant historical examples via input-dependent queries, trained end-to-end with the classifier using a score-based gradient estimator. This enables the full corpus of historical data to remain on an arbitrary filesystem during training and deployment. Experiments on synthetic benchmarks and Amazon Reviews '23 (electronics category) show improved robustness to distribution shift compared to standard classifiers, with VRAM scaling predictably as the length of the historical data sequence increases.
