Search papers, labs, and topics across Lattice.
This paper introduces VAIOM, a novel decoder-only Transformer designed for probabilistic next-return modeling in financial sequences, specifically using continuous multivariate financial-event vectors as input. By separating input representation from output likelihood, VAIOM effectively captures the numerical structure of financial data while employing categorical distributions for training and evaluation. The model outperforms a fixed single-bar LightGBM baseline across multiple test periods, demonstrating significant improvements in return likelihood and validation performance through the integration of continuous inputs and auxiliary objectives.
VAIOM achieves significant gains in financial sequence modeling by leveraging continuous inputs, outperforming traditional methods like LightGBM in return likelihood.
Financial observations are continuous, heterogeneous, and noisy, whereas decoder-only next-token models are usually built around discrete symbolic inputs. We introduce Vector-Input Autoregressive Inference for Ordinal-Return Modeling (VAIOM), a decoder-only Transformer for probabilistic next-return modeling on one-hour foreign-exchange bars. VAIOM separates input representation from output likelihood: continuous multivariate financial-event vectors preserve numerical structure at the input, while a categorical distribution over the next volatility-normalized return bucket supports cross-entropy training and likelihood evaluation. The selected 0.9M Hybrid Continuous Input model combines continuous event features with categorical asset metadata, a Mixture-of-Market-States return head, Gap, volatility-regime, and Ordinal auxiliary objectives, and full-sequence supervision. Models and preprocessing are fit using pre-2024 Train data; models are selected on 2024H2 Validation and evaluated without refitting on two 2025 Test periods. Across three independent training seeds, every model outperforms fixed single-bar LightGBM baseline in both Test halves. For the canonical checkpoint, paired gains over LightGBM are 0.029 and 0.043 bits per event. Validation experiments show that continuous input improves over discrete-token input under the same categorical return objective, full-sequence supervision improves over last-position training, and auxiliary representation shaping together with a mixture-structured return head improves return likelihood in controlled comparisons. A supporting capacity study finds that the smallest evaluated complete architecture rung achieves the strongest Validation likelihood on the present corpus.
