Lost in Visual Translation: A VLM-Assisted Perceptual-Semantic Coherence Framework for EEG-to-Image Reconstruction

2026-07-14

AI Summary

Current evaluation methods for EEG-to-image reconstruction struggle to disentangle visual fidelity from semantic recoverability, often overestimating reconstructions that are semantically accurate yet visually blurry. To address this limitation, this work proposes the BCI-Coherence Score (BCS), which introduces, for the first time, a perception–semantic consistency framework leveraging vision-language models (VLMs). The approach employs structured semantic probes to guide multiple VLMs in generating dual-dimensional tolerance-aware scores, which are then fused into a unified metric. Evaluated on T-PAS (MAE=0.079, r=0.700) and T-SAS (MAE=0.082, r=0.850), BCS significantly outperforms conventional metrics and demonstrates strong alignment with human judgments (Cohen’s κ=0.882), thereby establishing a new evaluation paradigm tailored for brain–computer interface–based image reconstruction.

Abstract

EEG-to-image evaluation should distinguish visual fidelity from recoverable meaning. Yet EEG-derived reconstructions are blurry, distorted, and low-detail, causing SSIM, LPIPS, and CLIP to penalize semantically recoverable outputs or reward plausible but incorrect ones. We analyze 6,855 ground-truth/reconstruction pairs from ATM, ENIGMA, BrainVis, and DreamDiffusion using semantic probes, caption harshness and blind-spot rates, and controlled degradations. Pixel metrics show near-zero correlation with semantic consistency, while representation metrics conflate perceptual and semantic errors. We therefore introduce a BCI-aware framework in which four VLMs assess image pairs through structured questions, producing Tolerant Perceptual Alignment Scores (T-PAS) and Tolerant Semantic Alignment Scores (T-SAS). Their consensus is distilled into the BCI-Coherence Score (BCS), a compact evaluator achieving a T-PAS MAE of 0.079 (r = 0.700) and a T-SAS MAE of 0.082 (r = 0.850) on our data. Human validation shows highly reliable joint coherence judgments, with Cohen's kappa = 0.882 +/- 0.174 and Krippendorff's alpha = 0.882, supporting perceptual-semantic recoverability over generic visual similarity. Code and resources are available at https://sukt03.github.io/BCS/.

Problem

• EEG-to-image reconstruction
• perceptual-semantic coherence
• evaluation metrics
• visual fidelity
• semantic consistency

Innovation

• EEG-to-image reconstruction
• perceptual-semantic coherence
• vision-language models
• BCI evaluation
• alignment scoring