Search papers, labs, and topics across Lattice.
This paper introduces a decision layer for streaming systems that optimally chooses between reusing existing expert models, spawning new ones, or deferring action based on incoming data. By framing the reuse and spawn decisions as one-sided sequential hypotheses with an indifference zone, the authors ensure statistically valid outcomes while maintaining finite-time anytime validity. The proposed method achieves zero false spawns and reuses on synthetic multi-concept streams, outperforming traditional heuristics in accuracy and reliability.
A novel decision-making framework for expert model management achieves zero false spawns and reuses, revolutionizing how streaming systems adapt to new data.
Streaming systems that maintain a pool of expert models must repeatedly decide whether to reuse an existing expert for arriving data, spawn a new one, or defer. We present a decision layer that makes all three outcomes statistically meaningful. Reuse and spawn are posed as one-sided sequential hypotheses on a conditional (mechanism-level) discrepancy, separated by an indifference zone; defer is exactly the state in which neither betting e-process has accumulated sufficient evidence. We prove finite-time anytime validity for the observable surrogate discrepancy of a predictable discriminator sequence, and an unconditional one-sided transfer to the population quantity in which each side's slack is the excess risk of a single discriminator; an empirically observed downward-bias regularity makes the spawn side exactly conservative. Recency without sacrificing the guarantee is obtained by a restarted e-detector: a bank of unwindowed betting supermartingales at geometrically spaced restart times (O(log t) memory), with the error budget spent over restart instances, which preserves lifetime anytime validity; spending over expert-creation order likewise controls multiplicity for unboundedly many experts. On synthetic multi-concept streams, Electricity, Covertype, and the recurrence-heavy INSECTS benchmark, the instance-accounted restarted bank achieves zero false spawns and zero false reuses after switches and matches or exceeds the retired windowed heuristic (INSECTS-reoccurring accuracy 0.675), making the deployed algorithm and the guaranteed algorithm one and the same.
