Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces a structural property of DynamicGate MLPs that allows for concurrent learning and inference by separating routing (gating) parameters from representation parameters. It mathematically formalizes sufficient conditions for this concurrency, proving that even with asynchronous updates, the inference output remains a valid model snapshot. This enables online adaptation and on-device learning without compromising inference stability.
DynamicGate MLPs unlock stable online learning by decoupling routing from representation, letting you tweak the model on the fly without crashing inference.
Conventional neural networks strictly separate learning and inference because if parameters are updated during inference, outputs become unstable and even the inference function itself is not well defined [1, 2, 3]. This paper shows that DynamicGate MLP structurally permits learning inference concurrency [4, 5]. The key idea is to separate routing (gating) parameters from representation (prediction) parameters, so that the gate can be adapted online while inference stability is preserved, or weights can be selectively updated only within the inactive subspace [4, 5, 6, 7]. We mathematically formalize sufficient conditions for concurrency and show that even under asynchronous or partial updates, the inference output at each time step can always be interpreted as a forward computation of a valid model snapshot [8, 9, 10]. This suggests that DynamicGate MLP can serve as a practical foundation for online adaptive and on device learning systems [11, 12].