Search papers, labs, and topics across Lattice.
This paper introduces a Multi-Rate Mixture-of-Experts (MR-MoE) framework that enhances Liquid Neural Networks (LNNs) for modeling complex multivariate time-series data. By employing multiple LNN-based experts operating at distinct time scales and incorporating adaptive gating and attention mechanisms, the framework effectively captures both fast-changing dynamics and slow-evolving trends. Experimental evaluations show that MR-MoE outperforms traditional models like LSTM and monolithic LNNs in terms of AUROC and AUPRC, while also ensuring computational efficiency.
Capturing diverse temporal dynamics with a single model is now feasible, thanks to a novel framework that combines multi-rate expert systems with adaptive attention.
Multivariate time-series data often exhibit complex temporal dependencies, irregular sampling, and heterogeneous dynamics across multiple time scales, making accurate sequence modeling particularly challenging. Traditional recurrent neural networks (RNNs), such as Long Short-Term Memory (LSTM) networks, operate in discrete time and may struggle to effectively capture continuous and irregular temporal behaviors. Liquid Neural Networks (LNNs) address some of these limitations through continuous-time dynamics, but standard LNN architectures typically rely on a single dynamical system, limiting their ability to model heterogeneous temporal patterns. To address these challenges, we propose a Multi-Rate Mixture-of-Experts (MR-MoE) framework built on top of Liquid Neural Networks. In the proposed architecture, multiple LNN-based experts operate at distinct time scales, enabling the model to explicitly separate fast-changing dynamics from slow-evolving temporal trends. A gating network further enables adaptive expert specialization based on input conditions. In addition, we incorporate both feature-level and temporal attention mechanisms to improve robustness, interpretability, and long-range dependency modeling. Feature-level attention suppresses noisy or irrelevant variables, while temporal attention selectively focuses on informative historical states. We evaluate the proposed framework on a complex multivariate time-series prediction task and compare it against strong baselines, including LSTM, monolithic LNN, and standard MoE models. Experimental results demonstrate that the proposed MR-MoE framework consistently achieves improved AUROC and AUPRC performance while maintaining favorable computational efficiency. These results highlight the effectiveness of combining continuous-time dynamics, multi-scale expert decomposition, and adaptive attention mechanisms for time-series modeling.
