CFM-Bench: A Unified Multi-Domain, Multi-Task Benchmark for Channel Foundation Models

2026-07-16

AI Summary

Existing foundational channel models lack a unified and fair evaluation benchmark, making cross-model comparisons difficult due to inconsistent assessment protocols. To address this gap, this work proposes CFM-Bench, the first standardized evaluation framework tailored for foundational channel models. CFM-Bench integrates six channel configurations and six task groups, spanning three key application domains: physical layer communications, radio access networks, and integrated sensing and communication. The benchmark combines 3GPP statistical simulations, ray-tracing data, real-world measurements, and multimodal vehicular datasets, employing trajectory-, session-, and link-level isolation to rigorously prevent test leakage and contamination from pretraining data. It supports diverse tasks including CSI feedback, channel extrapolation, beam prediction, and localization. CFM-Bench thus provides a reproducible and comparable platform that systematically advances research into the cross-domain generalization and practical deployment of foundational channel models.

Abstract

Channel foundation models (CFMs) are developing rapidly, with recent studies reporting benefits from pretraining across downstream wireless tasks. Yet CFMs are commonly evaluated in model-specific pipelines with different data, radio configurations, partitions, adaptation procedures, task definitions, and metrics. Reported comparisons therefore tend to show that pretraining improves over supervised training from scratch within one pipeline, but neither rank CFMs nor compare them fairly with task-specific models. We release CFM-Bench, a unified multi-domain, multi-task benchmark designed to address this gap. It curates six channel configurations spanning 3GPP statistical simulation, two independent ray-tracing pipelines, industrial and aerial measurements, and synchronized vehicular multimodal simulation. Official partitions isolate complete trajectories, measurement sessions, vehicle links, simulation realizations, or buffered spatial regions. CFM-Bench does not prescribe an external pretraining corpus or strategy; no benchmark split may be used for foundation-model pretraining, and the official training split is reserved exclusively for downstream fine-tuning. The benchmark additionally requires disclosure of all data used during model development and prohibits training-stage use of official test units. Six task groups are organized along three CFM application dimensions: physical-layer (PHY) channel intelligence, radio-access-network (RAN) decision intelligence, and integrated sensing and communication (ISAC). They cover CSI feedback, frequency and temporal channel extrapolation, propagation-state classification, current- and future-beam prediction, and single-frame and temporal localization. CFM-Bench provides a common substrate for comparing the transferability of channel representations across models, domains, and tasks.

Problem

• Channel Foundation Models
• benchmark
• multi-domain
• multi-task
• fair evaluation

Innovation

• Channel Foundation Models
• Unified Benchmark
• Multi-Domain Evaluation
• Data Partitioning
• Transferable Channel Representation