GraphGhost: Tracing Structures Behind Large Language Models

2025-10-07

AI Summary

To address the lack of structured interpretability in large language models’ (LLMs) reasoning mechanisms, this paper proposes GraphGhost—a unified graph-based modeling paradigm that formalizes neuron activations and signal propagation as directed weighted graphs. Leveraging graph algorithms such as PageRank, it enables cross-model quantitative analysis of shared and model-specific reasoning pathways. By identifying and intervening on critical neuron nodes, we provide the first empirical causal evidence linking specific graph structures to semantic reasoning capability: localized structural perturbations induce logical chain breakdown and semantic comprehension collapse. Integrating neural activation visualization, structured graph representation, and interpretable intervention, GraphGhost significantly enhances both the interpretability and controllability of LLMs’ internal reasoning architectures. This work establishes a novel paradigm for structure-aware model diagnosis and editing grounded in mechanistic graph principles.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Reasoning under Uncertainty: Graphical Models
• Natural Language Processing: (Large) Language Models

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Large Language Models (LLMs) demonstrate remarkable reasoning capabilities, yet the structural mechanisms underlying these abilities remain under explored. In this work, we introduce GraphGhost, a unified framework that represents neuron activations and their signal propagation as graphs, explaining how LLMs capture structural semantics from sequential inputs and generate outputs through structurally consistent mechanisms. This graph-based perspective enables us to employ graph algorithms such as PageRank to characterize the properties of LLMs, revealing both shared and model-specific reasoning behaviors across diverse datasets. We further identify the activated neurons within GraphGhost and evaluate them through structural interventions, showing that edits to key neuron nodes can trigger reasoning collapse, altering both logical flow and semantic understanding. Together, these contributions position GraphGhost as a powerful tool for analyzing, intervening in, and ultimately understanding the structural foundations of reasoning in LLMs.

Problem

• Unveils structural mechanisms behind LLM reasoning capabilities
• Explains how LLMs capture structural semantics from sequential inputs
• Identifies key neuron activations that control reasoning behaviors

Innovation

• GraphGhost models neuron activations as graphs
• It uses graph algorithms to analyze reasoning behaviors
• Structural interventions on neurons alter model reasoning