Hierarchical Frequency Tagging Probe (HFTP): A Unified Approach to Investigate Syntactic Structure Representations in Large Language Models and the Human Brain

2025-10-15

AI Summary

Existing methods struggle to localize syntactic computation modules within large language models (LLMs), hindering investigation into whether LLMs’ syntactic representations are brain-like. Method: We propose Hierarchical Frequency Tagging Probing (HFTP)—the first adaptation of frequency tagging to hierarchical syntactic structure analysis—enabling fine-grained, cross-modal probing at both single-neuron and cortical-region levels. Integrating intracranial electroencephalography (iEEG), neural representational similarity analysis (RSA), and multilayer perceptron (MLP) response modeling, we systematically compare syntactic encoding across GPT-2, Llama, Gemma, and human brain activity. Results: We find that LLMs encode syntax primarily in specific intermediate layers, whereas the human brain exhibits hierarchical functional specialization across left-hemisphere language regions. Moreover, as model scale increases, alignment between LLM representations and neural responses diverges rather than improves. This work establishes a novel paradigm for probing the cognitive mechanisms and neural interpretability of LLMs.

Technology Category

• Natural Language Processing: Syntax — Tagging, Chunking & Parsing
• Machine Learning: Large Multimodal Models (LMMs)
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Large Language Models (LLMs) demonstrate human-level or even superior language abilities, effectively modeling syntactic structures, yet the specific computational modules responsible remain unclear. A key question is whether LLM behavioral capabilities stem from mechanisms akin to those in the human brain. To address these questions, we introduce the Hierarchical Frequency Tagging Probe (HFTP), a tool that utilizes frequency-domain analysis to identify neuron-wise components of LLMs (e.g., individual Multilayer Perceptron (MLP) neurons) and cortical regions (via intracranial recordings) encoding syntactic structures. Our results show that models such as GPT-2, Gemma, Gemma 2, Llama 2, Llama 3.1, and GLM-4 process syntax in analogous layers, while the human brain relies on distinct cortical regions for different syntactic levels. Representational similarity analysis reveals a stronger alignment between LLM representations and the left hemisphere of the brain (dominant in language processing). Notably, upgraded models exhibit divergent trends: Gemma 2 shows greater brain similarity than Gemma, while Llama 3.1 shows less alignment with the brain compared to Llama 2. These findings offer new insights into the interpretability of LLM behavioral improvements, raising questions about whether these advancements are driven by human-like or non-human-like mechanisms, and establish HFTP as a valuable tool bridging computational linguistics and cognitive neuroscience. This project is available at https://github.com/LilTiger/HFTP.

Problem

• Identifying neural components encoding syntax in LLMs and brain
• Comparing syntactic processing mechanisms between AI and humans
• Investigating alignment of language model representations with brain activity

Innovation

• HFTP uses frequency-domain analysis for syntactic structure identification
• It probes both LLM neurons and human brain regions
• The tool compares model representations with cortical processing