Towards a Unified Textual Graph Framework for Spectral Reasoning via Physical and Chemical Information Fusion

2025-06-21

AI Summary

Existing spectral analysis methods suffer from overreliance on single-modality inputs, poor generalization—especially in zero-shot and few-shot settings—and limited interpretability. To address these limitations, we propose SpectraKG, the first multimodal spectral analysis framework integrating chemical knowledge graphs with large language models (LLMs). Our core innovation lies in unifying spectral data and molecular structures into a text-structured attributed graph, where “prompt nodes” explicitly encode physical measurement conditions and chemical semantics, and functional group priors are embedded to construct task-oriented textual graphs. This design enables LLMs to perform contextual reasoning over graph-structured representations, significantly enhancing zero-shot/few-shot generalization and decision interpretability. Extensive experiments demonstrate that SpectraKG achieves state-of-the-art performance across node-, edge-, and graph-level classification tasks.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Data Mining & Knowledge Management: Graph Mining, Social Network Analysis & Community
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

Motivated by the limitations of current spectral analysis methods-such as reliance on single-modality data, limited generalizability, and poor interpretability-we propose a novel multi-modal spectral analysis framework that integrates prior knowledge graphs with Large Language Models. Our method explicitly bridges physical spectral measurements and chemical structural semantics by representing them in a unified Textual Graph format, enabling flexible, interpretable, and generalizable spectral understanding. Raw spectra are first transformed into TAGs, where nodes and edges are enriched with textual attributes describing both spectral properties and chemical context. These are then merged with relevant prior knowledge-including functional groups and molecular graphs-to form a Task Graph that incorporates "Prompt Nodes" supporting LLM-based contextual reasoning. A Graph Neural Network further processes this structure to complete downstream tasks. This unified design enables seamless multi-modal integration and automated feature decoding with minimal manual annotation. Our framework achieves consistently high performance across multiple spectral analysis tasks, including node-level, edge-level, and graph-level classification. It demonstrates robust generalization in both zero-shot and few-shot settings, highlighting its effectiveness in learning from limited data and supporting in-context reasoning. This work establishes a scalable and interpretable foundation for LLM-driven spectral analysis, unifying physical and chemical modalities for scientific applications.

Problem

• Integrates knowledge graphs with LLMs for spectral analysis
• Unifies spectral and chemical data in Textual Graph format
• Enables interpretable, generalizable multi-modal spectral understanding

Innovation

• Integrates knowledge graphs with Large Language Models
• Unifies spectra and chemical data as Textual Graphs
• Uses Graph Neural Networks for multi-modal processing