Brain-Grasp: Graph-based Saliency Priors for Improved fMRI-based Visual Brain Decoding

2026-04-12

AI Summary

Existing fMRI-based visual brain decoding methods often neglect the spatial layout of salient objects, leading to reconstructed images that lack structural and semantic coherence. This work proposes a saliency-driven decoding framework that leverages graph-structured priors to translate structural cues from brain signals into spatial masks, which are then integrated with semantic embeddings to guide a single-stage frozen diffusion model for image reconstruction. Without fine-tuning the diffusion model, the approach enables lightweight, structure-aware decoding that preserves object fidelity while ensuring plausible natural scene composition. Experimental results demonstrate that the proposed method significantly improves both conceptual alignment and structural similarity between reconstructed images and original visual stimuli, offering a novel pathway toward efficient, interpretable, and structurally coherent brain decoding.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Humans and AI: Brain-Sensing and Analysis

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Recent progress in brain-guided image generation has improved the quality of fMRI-based reconstructions; however, fundamental challenges remain in preserving object-level structure and semantic fidelity. Many existing approaches overlook the spatial arrangement of salient objects, leading to conceptually inconsistent outputs. We propose a saliency-driven decoding framework that employs graph-informed saliency priors to translate structural cues from brain signals into spatial masks. These masks, together with semantic information extracted from embeddings, condition a diffusion model to guide image regeneration, helping preserve object conformity while maintaining natural scene composition. In contrast to pipelines that invoke multiple diffusion stages, our approach relies on a single frozen model, offering a more lightweight yet effective design. Experiments show that this strategy improves both conceptual alignment and structural similarity to the original stimuli, while also introducing a new direction for efficient, interpretable, and structurally grounded brain decoding.

Problem

• fMRI-based visual brain decoding
• object-level structure
• semantic fidelity
• saliency
• spatial arrangement

Innovation

• graph-based saliency priors
• fMRI-based visual decoding
• diffusion model
• structural preservation
• brain-guided image generation