Search papers, labs, and topics across Lattice.
This paper introduces CMP (Cognitive Memory Primitive), an architecture that utilizes sparse relational codes and a two-tier competitive memory system, learning through local, gradient-free updates without backpropagation. The authors hypothesize that catastrophic forgetting arises from the structural nature of backpropagation, and their experiments demonstrate that CMP exhibits 15-19 times better backward transfer compared to a Transformer model using online Elastic Weight Consolidation (EWC) across multiple text domains. Additionally, the findings highlight a significant accuracy gap favoring CMP over the Transformer baseline, while also addressing an unsuccessful attempt to integrate CMP with another accuracy-enhancing mechanism, emphasizing the value of transparent reporting of negative results.
Local, sparse learning can outperform backpropagation in resisting catastrophic forgetting, achieving up to 19 times better backward transfer in text domain tasks.
We introduce CMP (Cognitive Memory Primitive), an architecture that represents inputs as sparse relational codes, stores them in a two-tier competitive memory, and learns entirely through local, gradient-free updates, with no backpropagation anywhere in the network. We use this architecture to test a specific hypothesis: that catastrophic forgetting, usually treated as a training-time defect to be patched with replay or regularization, is instead a structural consequence of how backpropagation assigns credit and that a learning rule that is local and sparse by construction should resist it without a patch. On a controlled domain-incremental protocol across 15 text domains, three-seed replicated, CMP's backward transfer is 15-19x better than a matched-size Transformer trained with online EWC, and the result survives a domain-order control (reported as a range, +0.24 to +0.44, rather than a single figure). We report this alongside a real, substantial accuracy gap versus the Transformer baseline, a null result on a recognized vision benchmark, and a diagnosed, unresolved failure attempting to combine this architecture with a separate mechanism that improves raw accuracy, disclosed because an honest negative result is more useful than an omitted one. The central claim is narrow and falsifiable: local, sparse, non-backpropagation learning measurably resists catastrophic forgetting better than backpropagation with its standard fix, under conditions we state precisely.
