Search papers, labs, and topics across Lattice.
This paper introduces the Linear Reusable Neural Bases Architecture (LRNBA), which addresses the memory cost challenges associated with large AI models by using a shared set of neural bases to represent network blocks. By leveraging a design inspired by recurrent neural networks, LRNBA enables the construction of wider and deeper networks while maintaining parameter efficiency and stable training dynamics. Experimental results show that LRNBA achieves comparable or faster convergence and lower loss than traditional architectures, highlighting its potential for enhancing model performance within constrained resources.
Achieving significant network compression without sacrificing performance, LRNBA allows for the creation of deeper and wider models under the same parameter budget.
As large AI models become increasingly prevalent across a wide range of applications, memory cost has become a critical bottleneck in both training and inference. To mitigate this issue, we introduce the Linear Reusable Neural Bases Architecture (LRNBA), a novel framework aimed at improving parameter efficiency and reducing memory cost. Inspired by recurrent neural network (RNN) designs, the core idea of our approach is to represent each network block as a linear combination of a shared set of neural bases, thereby enjoying highly network compression rate while maintaining stable training. The proposed architecture allows for the construction of significantly wider and deeper networks under the same parameter budget. Extensive experiments demonstrate that our model achieves comparable or even faster convergence and lower loss than classical architectures, while maintaining stable training dynamics.
