Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
We introduce a derivative Gaussian Process (GP) that leverages gradient information with a novel two-directional budget per observed gradient, enabling more accurate surrogates than function-only GPs. This approach, integrated within a Vecchia approximation, reduces the computational cost of incorporating gradients from $md$ coordinates to $2m$ directional derivatives, yielding an $\mathcal{O}(m^3)$ factorization cost per prediction target. Empirical results demonstrate that this method achieves accuracy comparable to leading exact gradient-reduction techniques while significantly outperforming them in terms of time and memory efficiency, even surpassing function-only GP baselines.
Derivative GPs can now achieve state-of-the-art accuracy with less computation and memory than function-only baselines by cleverly budgeting gradient information.
Gradient observations promise more accurate Gaussian process (GP) surrogates, but the cost of incorporating them has long stood in the way of realizing that promise. We propose a derivative GP with a budget of just two directions per observed gradient. One direction focuses on each gradient's direct contribution to target prediction, while the other aggregates its indirect contributions through correlations with the conditioning function values. Within a Vecchia approximation, where each prediction conditions on $m$ nearby inputs in $d$ dimensions, this construction represents their $md$ gradient coordinates using at most $2m$ directional derivatives, giving $\mathcal{O}(m^3)$ dense factorization cost per prediction target. For general conditioning sets, we bound the posterior approximation error relative to using full gradients and characterize when the error is small or the approximation is exact. In simulations, our method matches the accuracy of a leading exact gradient-reduction method at equal conditioning set size. Because its cost grows much more slowly with that size, it can use conditioning sets well beyond the memory limit of the exact method, reaching lower prediction error with a small fraction of the time and memory. Notably, our method can exploit gradient observations while requiring less computation time or memory than function-only GP baselines.