Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper addresses catastrophic forgetting in Bayesian Binary Neural Networks (BBNNs) for continual learning on edge devices. They introduce BiMU, a bounded-memory variational inference approach that balances stability, plasticity, and forgetting by incorporating a controlled relaxation toward the prior and an uncertainty-dependent step size. Experiments on Permuted-MNIST and OpenLORIS-Object demonstrate that BiMU sustains learning, improves out-of-distribution detection, and reduces label queries/updates by up to 32x under class imbalance compared to standard BBNNs.
BBNNs can keep learning without catastrophic forgetting, and even save on compute, if you use the right variational objective.
Always-on edge systems must keep learning as conditions change under tight compute budgets and must detect unreliable predictions. Bayesian binary neural networks are attractive in this setting, but mean-field Bernoulli posteriors can saturate on long non-stationary streams, wiping out epistemic uncertainty and freezing plasticity. We propose BiMU, derived from a bounded-memory variational objective that balances stability, plasticity, and forgetting. BiMU combines a data term with controlled relaxation toward the prior and an uncertainty-dependent step size that prevents saturation and sustains informative uncertainty. This non-degenerate posterior enables fully online, buffer-free active querying via Monte Carlo disagreement, reducing label queries and backpropagation updates under imbalance. BiMU sustains learning and strong OOD detection on 1000-tasks Permuted-MNIST, and on OpenLORIS-Object achieves up to 32$\times$ label/update savings at matched accuracy under class imbalance and feature compression.