Search papers, labs, and topics across Lattice.
This paper introduces a hardware-oriented methodology designed to accelerate discrete Bayesian inference on resource-constrained edge devices by optimizing variational message-passing algorithms. By restructuring memory layouts and implementing tensor-clustering strategies, the authors achieve significant reductions in latency and memory footprint, resulting in optimized algorithms for Hidden Markov Models. Benchmarks on an NVIDIA Jetson Orin AGX demonstrate speedups of up to 5x, with consistent output quality compared to baseline methods, highlighting the practical viability of Bayesian inference in edge computing environments.
Achieving up to 5x speedups in Bayesian inference on edge devices could revolutionize real-time decision-making in resource-limited settings.
Bayesian inference provides a principled foundation for reasoning under uncertainty, but its computational cost hinders deployment on resource-constrained edge devices. In this paper, we present a hardware-oriented methodology for accelerating discrete Bayesian inference on commercial off-the-shelf embedded GPUs. We identify that the latency of a broad class of variational message-passing algorithms is dominated by tensor contractions. Our approach restructures the memory layout of these operations using two complementary merging strategies that produce compact, regularly-shaped primitives better suited for efficient GPU execution. We then introduce optional sparse array representations and a tensor-clustering scheme to reduce the memory footprint. We instantiate the methodology and produce optimized variants of three message-passing algorithms for Hidden Markov Models (HMMs), namely variational filtering, variational message passing, and marginal message passing. Furthermore, we complement this with a machine-learning-based autotuner that automatically selects the best-performing algorithmic variant for a given generative model specification. Benchmarked on an NVIDIA Jetson Orin AGX across 770 randomly sampled realistic Partially Observable Markov Decision Process (POMDP) configurations, our implementations achieve speedups of up to 5x, with typical gains of 2-2.5x, while producing numerically identical outputs to the baseline implementations.
