Search papers, labs, and topics across Lattice.
The paper introduces Continual Learning Adaptive Normalization (CLeAN), a novel normalization technique for continual learning of tabular data in dynamic environments. CLeAN uses learnable parameters updated via an Exponential Moving Average (EMA) to estimate global feature scales, enabling adaptation to evolving data distributions without requiring access to the entire dataset. Experiments on two datasets and with multiple continual learning strategies (Reservoir Experience Replay, A-GEM, EwC) show that CLeAN improves performance on new data and reduces catastrophic forgetting.
A simple adaptive normalization technique can significantly improve continual learning performance on tabular data by mitigating catastrophic forgetting in dynamic environments.
Artificial intelligence systems predominantly rely on static data distributions, making them ineffective in dynamic real-world environments, such as cybersecurity, autonomous transportation, or finance, where data shifts frequently. Continual learning offers a potential solution by enabling models to learn from sequential data while retaining prior knowledge. However, a critical and underexplored issue in this domain is data normalization. Conventional normalization methods, such as min-max scaling, presuppose access to the entire dataset, which is incongruent with the sequential nature of continual learning. In this paper we introduce Continual Learning Adaptive Normalization (CLeAN), a novel adaptive normalization technique designed for continual learning in tabular data. CLeAN involves the estimation of global feature scales using learnable parameters that are updated via an Exponential Moving Average (EMA) module, enabling the model to adapt to evolving data distributions. Through comprehensive evaluations on two datasets and various continual learning strategies, including Resevoir Experience Replay, A-GEM, and EwC we demonstrate that CLeAN not only improves model performance on new data but also mitigates catastrophic forgetting. The findings underscore the importance of adaptive normalization in enhancing the stability and effectiveness of tabular data, offering a novel perspective on the use of normalization to preserve knowledge in dynamic learning environments.
