Multivariate Stock Time-Series Forecasting Using an Improved Transformer

As financial markets become increasingly digital and intelligent, stock-price forecasting has emerged as an important topic in financial engineering, artificial intelligence, and data mining. Stock prices are shaped by firm fundamentals, macroeconomic conditions, investor sentiment, and market policies, while also exhibiting strong nonlinearity, nonstationarity, and noise. Extracting useful information from historical financial data and developing accurate and stable forecasting models therefore remain challenging. This study proposes an improved Transformer for multivariate stock time-series forecasting that combines feature-gated embedding with a residual prediction head. Daily data for Apple (AAPL), Microsoft (MSFT), and Google (GOOG) were obtained from the public YFinance interface for the period from January 1, 2000, to January 1, 2020. The model uses eight features—Open, High, Low, Close, Volume, RSI, ATR, and MACD—from the previous 30 trading days to predict the closing price five trading days ahead. A strictly chronological split, training-set-only normalization, and separate sliding-window construction are adopted to prevent look-ahead leakage. The proposed model is compared with Persistence, residual XGBoost, LSTM, a vanilla Transformer, and a standard feature-gated Transformer through the main experiment, cross-stock generalization tests, an ablation study, and a supplementary return-prediction experiment. On AAPL, the model achieves an RMSE of 1.2712, an MAE of 0.9242, and a MAPE of 2.5268%, slightly outperforming Persistence and residual XGBoost. It performs nearly identically to Persistence on GOOG and is slightly weaker than Persistence and residual XGBoost on MSFT, while consistently outperforming the LSTM and Transformer-based deep-learning baselines. These results demonstrate the accuracy and stability of the proposed model.