$\mathcal{S}^2$IT: Stepwise Syntax Integration Tuning for Large Language Models in Aspect Sentiment Quad Prediction

2026-04-25

AI Summary

This work addresses the challenge that large language models struggle to effectively capture syntactic structures in aspect-based sentiment quadruple prediction. To overcome this limitation, the authors propose a stepwise syntax-fusion fine-tuning framework that decomposes quadruple generation into two stages: global syntax-guided extraction followed by local syntax-guided classification. A three-stage fine-tuning strategy is introduced, progressively injecting fine-grained syntactic knowledge through dependency tree reconstruction, element linking prediction, and node classification tasks. This approach represents the first systematic integration of both global and local syntactic information into generative large language models, substantially enhancing their capacity to model complex syntactic dependencies. The method achieves state-of-the-art performance across multiple benchmark datasets, setting a new performance standard for this task.

Technology Category

• Natural Language Processing: Syntax — Tagging, Chunking & Parsing
• Machine Learning: Large Multimodal Models (LMMs)
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Large language models for search
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems

Abstract

Aspect Sentiment Quad Prediction (ASQP) has seen significant advancements, largely driven by the powerful semantic understanding and generative capabilities of large language models (LLMs). However, while syntactic structure information has been proven effective in previous extractive paradigms, it remains underutilized in the generative paradigm of LLMs due to their limited reasoning capabilities. In this paper, we propose S^2IT, a novel Stepwise Syntax Integration Tuning framework that progressively integrates syntactic structure knowledge into LLMs through a multi-step tuning process. The training process is divided into three steps. S^2IT decomposes the quadruple generation task into two stages: 1) Global Syntax-guided Extraction and 2) Local Syntax-guided Classification, integrating both global and local syntactic structure information. Finally, Fine-grained Structural Tuning enhances the model's understanding of syntactic structures through the prediction of element links and node classification. Experiments demonstrate that S^2IT significantly improves state-of-the-art performance across multiple datasets. Our implementation will be open-sourced at https://github.com/DMIRLAB-Group/S2IT.

Problem

• Aspect Sentiment Quad Prediction
• Large Language Models
• Syntactic Structure
• Generative Paradigm
• Syntax Integration

Innovation

• Stepwise Syntax Integration
• Aspect Sentiment Quad Prediction
• Syntactic Structure
• Large Language Models
• Fine-grained Structural Tuning