Toward Realistic Wi-Fi Fault Diagnosis: A Multi-Modal Benchmark

2026-05-21

AI Summary

Diagnosing Wi-Fi failures in heterogeneous environments remains challenging due to the underutilization of multimodal operational data. This work addresses this gap by constructing a real-world campus Wi-Fi testbed and collecting over 10,000 multimodal fault samples across diverse wireless scenarios through automated fault injection. The study introduces the first publicly available Wi-Fi failure dataset that simultaneously encompasses cross-layer heterogeneous observations and establishes a unified multitask, multimodal diagnostic benchmark. Furthermore, it proposes a reasoning consistency evaluation framework tailored for large language models (LLMs). Experimental results demonstrate that existing methods struggle to effectively fuse heterogeneous data, whereas LLM-based approaches show promising potential in achieving consistent diagnostic reasoning, thereby highlighting a critical direction for advancing multimodal fault diagnosis.

Abstract

Intelligent network operation and maintenance systems in modern networks continuously generate large volumes of multi-modal operational data. However, Wi-Fi fault diagnosis under heterogeneous operational environments remains insufficiently understood. We build a real-world Wi-Fi testbed deployed in campus working environments with an automated fault injection system, and collect a multi-modal Wi-Fi fault dataset containing over 10,000 fault samples across diverse wireless scenarios. To the best of our knowledge, this is among the first publicly available datasets jointly capturing heterogeneous cross-layer operational observations for Wi-Fi fault diagnosis. Based on this dataset, we establish a unified benchmark spanning multiple diagnosis tasks, operational modalities, and representative diagnosis paradigms. Experimental results indicate that effectively leveraging heterogeneous operational data remains challenging for existing diagnosis approaches. We further evaluate emerging LLM-based approaches and develop a reasoningoriented evaluation framework to assess the consistency between generated diagnostic analyses and actual network conditions. Our findings suggest several important considerations for future multi-modal Wi-Fi diagnosis.

Problem

• Wi-Fi fault diagnosis
• heterogeneous environments
• multi-modal data
• network operation and maintenance
• cross-layer observations

Innovation

• multi-modal dataset
• Wi-Fi fault diagnosis
• automated fault injection
• cross-layer observations
• LLM-based reasoning evaluation