Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces Flow-Corrected Thompson Sampling (fcTS), a Bayesian approach designed to address the challenges of non-stationary linear contextual bandits by effectively reusing historical data through a drift model. By incorporating confidence weights that reflect the reliability of transported observations, fcTS adapts to linear parameter drift, periodic variations, and regime switches, allowing for efficient posterior updates. The method significantly outperforms traditional forgetting-based strategies in scenarios with structured non-stationarity, demonstrating a marked improvement in sample efficiency by correcting and reweighting past rewards rather than discarding them.
Correcting and reweighting historical observations in non-stationary bandits can yield substantial sample efficiency gains, outperforming conventional methods that simply forget past data.
We study non-stationary linear contextual bandits where the reward model drifts over time, rendering classical contextual bandit algorithms brittle because historical data becomes systematically biased. We propose Flow-Corrected Thompson Sampling (fcTS), a Bayesian method that reuses experience by transporting past rewards to the present using an explicit drift model and incorporating each transported observation with a confidence weight that reflects transport reliability. This yields a unified template that specializes in (i) linear parameter drift via online slope estimation and reward correction, (ii) periodic variation via phase-aware reuse across cycles, and (iii) recurring regime switches via changepoint detection and regime-specific posterior memory. The resulting posterior updates remain closed-form under a linear Gaussian model and can be implemented efficiently with truncated, incrementally updated sufficient statistics. Across five controlled case studies and a semi-synthetic portfolio-selection benchmark with multiple overlapping non-stationarities, fcTS outperforms standard forgetting-based baselines (discounting, sliding windows, and periodic restarts), with the largest gains in settings exhibiting recurring temporal structure. These results demonstrate that when non-stationarity is structured, correcting and reweighting historical observations can be substantially more sample-efficient than uniformly discarding them.