Search papers, labs, and topics across Lattice.
This paper introduces a novel training method for Boolean neural networks where node values are strictly $\pm 1$ and weights are typically also $\pm 1$. The method replaces loss minimization with a non-convex constraint formulation enforcing local Boolean threshold function (BTF) consistency and architectural concurrence. The reflect-reflect-relax (RRR) projection algorithm reconciles these constraints, achieving exact solutions or strong generalization across tasks like multiplier-circuit discovery and logic-network inference, where standard gradient-based methods struggle.
Forget gradient descent: this new projection-based method trains neural networks with Boolean data to discover exact solutions and strong generalization in tasks where standard methods fail.
We develop a method for training neural networks on Boolean data in which the values at all nodes are strictly $\pm 1$, and the resulting models are typically equivalent to networks whose nonzero weights are also $\pm 1$. The method replaces loss minimization with a nonconvex constraint formulation. Each node implements a Boolean threshold function (BTF), and training is expressed through a divide-and-concur decomposition into two complementary constraints: one enforces local BTF consistency between inputs, weights, and output; the other imposes architectural concurrence, equating neuron outputs with downstream inputs and enforcing weight equality across training-data instantiations of the network. The reflect-reflect-relax (RRR) projection algorithm is used to reconcile these constraints. Each BTF constraint includes a lower bound on the margin. When this bound is sufficiently large, the learned representations are provably sparse and equivalent to networks composed of simple logical gates with $\pm 1$ weights. Across a range of tasks -- including multiplier-circuit discovery, binary autoencoding, logic-network inference, and cellular automata learning -- the method achieves exact solutions or strong generalization in regimes where standard gradient-based methods struggle. These results demonstrate that projection-based constraint satisfaction provides a viable and conceptually distinct foundation for learning in discrete neural systems, with implications for interpretability and efficient inference.
