Search papers, labs, and topics across Lattice.
This paper explores the limitations of current machine learning approaches in continual learning and proposes a novel architecture called 4MAS (4 Module Awake/Sleep) that mimics biological learning processes by utilizing asymmetric hemispheres with distinct long- and short-term memory mechanisms. The framework incorporates concepts such as experience replay and REM sleep to enhance memory consolidation during incremental learning tasks. Experimental results demonstrate that 4MAS achieves competitive accuracy on Split-MNIST, Split-Fashion-MNIST, and Split-CIFAR-100 datasets, indicating its potential for improving lifelong learning in neural models.
4MAS leverages asymmetric hemispheric structures and sleep-like periods to significantly enhance memory retention in continual learning tasks.
Persistent intelligent systems require the ability to learn continually, but current machine learning approaches face significant challenges in this area compared to biological learning systems. Machine learning algorithms typically trade off retention of previously learned information and adaptation to new or changing data patterns. When continual learning capabilities are absent, algorithms must undergo retraining using the entire data set, an approach that becomes impractical when original training data are unavailable due to storage constraints, financial or computational costs, or privacy restrictions. However, biological animals can learn continually, without experiencing catastrophic forgetting. This paper attempts to build a high-level framework for how animals learn and preserve knowledge by modelling neural components and states that are known to be related to memory consolidation. We focus on three concepts: experience replay, REM sleep, and bilaterality. We propose 4MAS (4 Module Awake/Sleep), a novel macroarchitecture demonstrating how machine learning models might benefit from asymmetric hemispheres, each with their own long- and short-term memory mechanisms, and how a period of sleep between incremental learning tasks might benefit memory consolidation. Finally, we present results showing that our architecture achieves competitive results on the Split-MNIST, Split-Fashion-MNIST and Split-CIFAR-100 datasets, with 98.3%, 84.9%, and 29.29% accuracy respectively.
