Search papers, labs, and topics across Lattice.
This paper studies the performance of majority-vote ensembles when training data exhibits Markov dependence, establishing an information-theoretic lower bound of $\Omega(\sqrt{\Tmix/n})$ on the excess classification risk. It demonstrates that dependence-agnostic uniform bagging is suboptimal, exhibiting an $\Omega(\Tmix/\sqrt{n})$ excess risk. The authors then introduce adaptive spectral routing, which partitions data using the empirical Fiedler eigenvector and achieves the minimax rate of $\mathcal{O}(\sqrt{\Tmix/n})$ on a graph-regular subclass.
Uniform bagging's performance degrades significantly with Markov-dependent data, but a novel spectral routing algorithm can restore minimax optimality without knowledge of the mixing time.
Majority-vote ensembles achieve variance reduction by averaging over diverse, approximately independent base learners. When training data exhibits Markov dependence, as in time-series forecasting, reinforcement learning (RL) replay buffers, and spatial grids, this classical guarantee degrades in ways that existing theory does not fully quantify. We provide a minimax characterization of this phenomenon for discrete classification in a fixed-dimensional Markov setting, together with an adaptive algorithm that matches the rate on a graph-regular subclass. We first establish an information-theoretic lower bound for stationary, reversible, geometrically ergodic chains in fixed ambient dimension, showing that no measurable estimator can achieve excess classification risk better than $\Omega(\sqrt{\Tmix/n})$. We then prove that, on the AR(1) witness subclass underlying the lower-bound construction, dependence-agnostic uniform bagging is provably suboptimal with excess risk bounded below by $\Omega(\Tmix/\sqrt{n})$, exhibiting a $\sqrt{\Tmix}$ algorithmic gap. Finally, we propose \emph{adaptive spectral routing}, which partitions the training data via the empirical Fiedler eigenvector of a dependency graph and achieves the minimax rate $\mathcal{O}(\sqrt{\Tmix/n})$ up to a lower-order geometric cut term on a graph-regular subclass, without knowledge of $\Tmix$. Experiments on synthetic Markov chains, 2D spatial grids, the 128-dataset UCR archive, and Atari DQN ensembles validate the theoretical predictions. Consequences for deep RL target variance, scalability via Nystr\"om approximation, and bounded non-stationarity are developed as supporting material in the appendix.
