Search papers, labs, and topics across Lattice.
EnTransformer, a novel deep generative forecasting framework, combines engression (a stochastic learning paradigm) with Transformers to model complex joint predictive distributions in multivariate time series. By injecting stochastic noise and optimizing an energy-based scoring objective, EnTransformer learns the conditional predictive distribution without parametric assumptions. Experiments on benchmark datasets (Electricity, Traffic, Solar, etc.) show EnTransformer outperforms existing models in producing well-calibrated probabilistic forecasts.
Ditch restrictive parametric assumptions in multivariate time series forecasting: EnTransformer uses a deep generative approach to directly learn complex joint predictive distributions, outperforming benchmarks.
Reliable uncertainty quantification is critical in multivariate time series forecasting problems arising in domains such as energy systems and transportation networks, among many others. Although Transformer-based architectures have recently achieved strong performance for sequence modeling, most probabilistic forecasting approaches rely on restrictive parametric likelihoods or quantile-based objectives. They can struggle to capture complex joint predictive distributions across multiple correlated time series. This work proposes EnTransformer, a deep generative forecasting framework that integrates engression, a stochastic learning paradigm for modeling conditional distributions, with the expressive sequence modeling capabilities of Transformers. The proposed approach injects stochastic noise into the model representation and optimizes an energy-based scoring objective to directly learn the conditional predictive distribution without imposing parametric assumptions. This design enables EnTransformer to generate coherent multivariate forecast trajectories while preserving Transformers'capacity to effectively model long-range temporal dependencies and cross-series interactions. We evaluate our proposed EnTransformer on several widely used benchmarks for multivariate probabilistic forecasting, including Electricity, Traffic, Solar, Taxi, KDD-cup, and Wikipedia datasets. Experimental results demonstrate that EnTransformer produces well-calibrated probabilistic forecasts and consistently outperforms the benchmark models.
