Search papers, labs, and topics across Lattice.
This paper introduces CFM-Bench, a comprehensive benchmark for evaluating channel foundation models (CFMs) across multiple domains and tasks, addressing the inconsistencies in current evaluation methodologies. By curating diverse channel configurations and enforcing strict guidelines on data usage and model training, CFM-Bench enables fair comparisons between CFMs and task-specific models. The benchmark encompasses six task groups related to physical-layer intelligence, radio-access-network decision-making, and integrated sensing and communication, facilitating a more standardized assessment of model performance and transferability.
CFM-Bench reveals that without a unified evaluation framework, the true potential of channel foundation models remains obscured, hindering meaningful comparisons with task-specific architectures.
Channel foundation models (CFMs) are developing rapidly, with recent studies reporting benefits from pretraining across downstream wireless tasks. Yet CFMs are commonly evaluated in model-specific pipelines with different data, radio configurations, partitions, adaptation procedures, task definitions, and metrics. Reported comparisons therefore tend to show that pretraining improves over supervised training from scratch within one pipeline, but neither rank CFMs nor compare them fairly with task-specific models. We release CFM-Bench, a unified multi-domain, multi-task benchmark designed to address this gap. It curates six channel configurations spanning 3GPP statistical simulation, two independent ray-tracing pipelines, industrial and aerial measurements, and synchronized vehicular multimodal simulation. Official partitions isolate complete trajectories, measurement sessions, vehicle links, simulation realizations, or buffered spatial regions. CFM-Bench does not prescribe an external pretraining corpus or strategy; no benchmark split may be used for foundation-model pretraining, and the official training split is reserved exclusively for downstream fine-tuning. The benchmark additionally requires disclosure of all data used during model development and prohibits training-stage use of official test units. Six task groups are organized along three CFM application dimensions: physical-layer (PHY) channel intelligence, radio-access-network (RAN) decision intelligence, and integrated sensing and communication (ISAC). They cover CSI feedback, frequency and temporal channel extrapolation, propagation-state classification, current- and future-beam prediction, and single-frame and temporal localization. CFM-Bench provides a common substrate for comparing the transferability of channel representations across models, domains, and tasks.
