EyeLayer: Integrating Human Attention Patterns into LLM-Based Code Summarization

2026-02-25

AI Summary

This work proposes EyeLayer, a lightweight attention-augmentation module designed to incorporate human eye-tracking attention patterns during code reading into large language models to enhance code summarization. For the first time, developer eye-tracking data is leveraged as prior knowledge in the code summarization task. EyeLayer employs a multimodal Gaussian mixture model to redistribute attention over code token embeddings, guiding the model to focus on semantically critical regions. Notably, it requires no modification to the underlying model architecture and exhibits cross-model compatibility. Experiments across mainstream models—including LLaMA-3.2, Qwen3, and CodeBERT—demonstrate consistent improvements over strong baselines, with BLEU-4 scores increasing by up to 13.17%, thereby validating the efficacy of human attention signals in augmenting code semantic understanding.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Large Vision Models
• Natural Language Processing: Summarization

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Large language models for search
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Code summarization is the task of generating natural language descriptions of source code, which is critical for software comprehension and maintenance. While large language models (LLMs) have achieved remarkable progress on this task, an open question remains: can human expertise in code understanding further guide and enhance these models? We propose EyeLayer, a lightweight attention-augmentation module that incorporates human eye-gaze patterns, as a proxy of human expertise, into LLM-based code summarization. EyeLayer models human attention during code reading via a Multimodal Gaussian Mixture, redistributing token embeddings based on learned parameters (μ_i, σ_i^2) that capture where and how intensively developers focus. This design enables learning generalizable attention priors from eye-tracking data and incorporating them into LLMs seamlessly, without disturbing existing representations. We evaluate EyeLayer across diverse model families (i.e., LLaMA-3.2, Qwen3, and CodeBERT) covering different scales and architectures. EyeLayer consistently outperforms strong fine-tuning baselines across standard metrics, achieving gains of up to 13.17% on BLEU-4. These results demonstrate that human gaze patterns encode complementary attention signals that enhance the semantic focus of LLMs and transfer effectively across diverse models for code summarization.

Problem

• code summarization
• human attention
• eye-tracking
• large language models
• software comprehension

Innovation

• EyeLayer
• human attention patterns
• eye-tracking
• code summarization
• attention augmentation