Search papers, labs, and topics across Lattice.
Averaging losses across channels in multivariate time-series forecasting masks severe gradient conflict鈥攐ver 30% of variable-wise gradients negatively align, causing more than half of variables to underperform a single-target oracle. To resolve this without expensive multi-pass gradient surgery, the authors introduce Per-Variable Surgery (PV-Surgery), an optimizer-side intervention that computes single-pass gradient proxies, isolates reliable layers, and conditionally pools conflicting variable updates. Across five architectures and seven benchmarks, PV-Surgery consistently mitigates negative transfer, reducing average MSE by 3.61% and MAE by 2.93% across multiple forecasting horizons.
Nearly a third of variable-wise gradients directly oppose each other during standard multivariate forecasting, but resolving these conflicts via single-pass gradient surgery reliably boosts accuracy without multiple backward passes.
In data-driven training, multivariate time-series forecasting is usually optimized with a scalar loss averaged over samples, variables, and horizons. This averaging is convenient, but the optimizer sees only the aggregated gradient, which does not reveal whether the variable-wise contributions align or oppose one another. To quantify how often this disagreement arises, we measure the variable-wise gradients directly and find that 30.6% of their pairwise cosine similarities are negative on average across seven datasets. However, conflict and harm are not the same thing. Under shared training 35 of the 64 variables do worse than a full-input single-target oracle, and the harmed fraction is not reliably predicted by how often gradients conflict. We propose Per-Variable Surgery (PV-Surgery), an optimizer-side training strategy for backbones with cache-compatible layers. One backward pass builds variable-wise gradient proxies from output-side signals and keeps the pointwise forecasting loss. Reliability-aware selection targets layers whose proxy sums closely approximate their shared-gradient slices. Conditional pooling forms anchor and conflict pools without dropping variables. Common-direction surgery aligns variable or pooled gradients with their normalized mean and restores input norms to avoid reweighting. In experiments across five backbones, seven datasets, and four horizons, PV-Surgery lowers MSE by 3.61% and MAE by 2.93% on average. For multivariate forecasting, this indicates that the variable-wise structure hidden by mean-loss training is a usable optimization signal.
