Search papers, labs, and topics across Lattice.
This paper introduces ORBIT (Omni-Range Bootstrap Incremental Training), a novel training regime for time series foundation models (TSFMs) that addresses the challenges of domain imbalance, context requirements, and prediction horizons in large-scale heterogeneous datasets. By integrating Bootstrap Multi-Level Sampling and Omni-Range Incremental Training, ORBIT allows for explicit control over the training distribution, enhancing the model's ability to handle missing data and varying context lengths. The Falcon-2.0 model, trained under this regime, exhibits impressive zero-shot forecasting capabilities across multiple domains, outperforming existing benchmarks on GIFT-Eval and fev-bench.
ORBIT enables unprecedented control over training distributions, leading to superior zero-shot forecasting performance in time series models.
Time series foundation models (TSFMs) have advanced primarily through architectural innovation, while training regimes for large-scale heterogeneous corpora remain under-explored. As a result, pre-training distributions are often poorly controlled with respect to domain imbalance, context requirements, prediction horizons, and missingness. We introduce ORBIT (Omni-Range Bootstrap Incremental Training), a training paradigm that makes this distribution explicit and controllable. ORBIT combines Bootstrap Multi-Level Sampling, which controls dataset exposure and samples records, target variables, context windows, and prediction horizons, with Omni-Range Incremental Training, which varies context lengths and prediction horizons throughout a single training stage. Under ORBIT, we train Falcon-2.0, a simple univariate encoder-only Transformer with missingness-aware triple-channel patch tokenization and parallel patch prediction. We further introduce Rank-Guided Cross-Depth Alignment, a training objective that uses late-layer representations as stop-gradient teachers for shallow layers without additional inference cost. Evaluations on GIFT-Eval and fev-bench demonstrate strong zero-shot forecasting performance across diverse domains and frequencies.
