Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper decomposes Chronos-2's group attention mechanism into its V/O (pooling) and Q/K (weighting) components to understand their contributions to multivariate and in-context learning (ICL) forecasting. Uniform pooling (V/O only) consistently benefits performance across most configurations, whereas learned weighting (Q/K) degrades ICL performance significantly in many cases. Specifically, uniforming attention in the first block alone improves all tested ICL configurations.
Learned attention weights actively harm in-context learning performance in Chronos-2, while simple uniform pooling provides consistent benefits.
Group attention, introduced by the time series forecasting model Chronos-2, attends over the variates of a group at a fixed patch index and serves both multivariate (MV) and in-context learning (ICL) forecasting. Rather than evaluating this cross-variate attention design as a whole, we ask which part of the mechanism earns the benefit and probe its applicability to both MV and ICL regimes. By editing the attention matrix $\alpha$ at inference we separate the two pathways a head comprises: V/O, which projects a weighted summary of the group, and Q/K, which decides the weights. Uniform pooling (V/O without any Q/K weighting) is positive on 18 of our 20 sensor-network configurations, while the learned weighting (Q/K) splits by group type: its contribution is positive or negligible for MV, but materially degrades 8 of the 10 sensor-network ICL configurations, leaving 4 of them worse than univariate inference. By isolating the impact of different layers, we find that uniforming $\alpha$ in the first block alone improves every ICL configuration we test.