Search papers, labs, and topics across Lattice.
This paper conducts a systematic break-even analysis to determine when deploying foundation models for time series forecasting is justified compared to classical methods like XGBoost. By evaluating 30 benchmark datasets and comparing zero-shot and LoRA fine-tuned foundation models against traditional baselines, the authors find that foundation models outperform classical methods on 15 datasets across various training set sizes. Notably, they establish a decision framework that allows practitioners to quickly assess deployment strategies based on dataset characteristics, revealing that under certain conditions, classical methods can outperform foundation models even with minimal training data.
Foundation models can decisively outperform classical methods in time series forecasting, but only under specific data conditions鈥攌nowing when to deploy them is crucial for efficiency.
Deploying a time series foundation model requires GPU infrastructure, engineering overhead, and carries no guarantee of improvement over XGBoost. We provide the first systematic break-even analysis answering when this investment pays off. Across 30 benchmark datasets, we compare zero-shot and LoRA fine-tuned foundation models (Chronos, Moirai, Lag-Llama) against classical baselines (Naive, ETS, ARIMA, XGBoost) at six training set sizes from 2% to 100% of available data. Foundation models outperform classical methods at every evaluated training fraction on 15 of 30 datasets -- GPU deployment is unconditionally justified on these regardless of data volume. On 6 datasets, classical methods surpass zero-shot foundation models with as little as 2% of training data (21-2,768 samples); on the remaining 9, break-even ranges from 24 to 8,361 samples. One robust deployment rule requires no model training: if n_train < 700 and seasonality is non-negligible, use FM zero-shot and skip fine-tuning -- this resolves 10 of 30 deployment decisions immediately. Contrary to common practice, LoRA fine-tuning can actively degrade performance on short series. We operationalise these findings as a two-step decision framework -- compute dataset length and seasonality strength, run a brief 5-10% pilot only if needed -- enabling practitioners to make the FM-versus-classical decision before committing to full infrastructure. Four dataset features motivate mechanistic hypotheses for the remaining cases, though reliable automated prediction at this benchmark scale remains an open problem. Code, benchmark, and decision tools are available at https://github.com/nicolaisi/fm-breakeven.
