Search papers, labs, and topics across Lattice.
This paper introduces WALoMA, a multitask wireless foundation model that utilizes adaptive low-rank masked autoencoders to overcome the limitations of specialized deep learning models in sixth-generation (6G) wireless applications. By employing a self-supervised learning approach on unlabeled channel state information (CSI), WALoMA achieves impressive performance across five key tasks, demonstrating a composite score of 87.80% while using only 14.68% of the parameters compared to a large wireless model baseline. The model's innovative architecture, which includes 2D positional encoding and low-rank adaptation, allows it to effectively learn transferable representations, even with limited labeled data.
WALoMA outperforms traditional models by achieving an 87.80% composite score while using just 14.68% of the parameters, redefining efficiency in wireless task performance.
This paper proposes a multitask wireless foundation model via adaptive low-rank masked autoencoders (WALoMA), a unified multi-task foundation model for sixth-generation (6G) wireless physical layer architectures, to address the limitations of specialized, task-specific deep learning models and the practical challenge of scarce labeled wireless datasets. By leveraging concepts inspired by foundation models, the proposed framework adopts a masked autoencoder (MAE) paradigm to learn from unlabeled channel data, to significantly reduce reliance on extensive annotations. The model treats wireless channel state information (CSI) as a universal modality and learns transferable representations through self-supervised channel reconstruction. Key architectural novelties include the use of 2D positional encoding (PE) to explicitly preserve the spatial-frequency relationships between antennas and subcarriers, and low-rank adaptation (LoRA) for parameter-efficient fine-tuning. The framework's efficacy is demonstrated across five downstream tasks, achieving individual scores of 96.47\% for LoS/NLoS classification, 80.45\% for beam prediction, 85.78\% for channel interpolation, 99.12\% for channel estimation, and 77.18\% for channel charting. Consequently, numerical results show that the proposed model achieves a composite score of 87.80\%, significantly outperforming the 59.90\% achieved by the large wireless model (LWM) baseline while training an average of only 14.68\% of total parameters, and maintaining strong performance even under extremely limited labeled data conditions.
