LLMs Can Also Do Well! Breaking Barriers in Semantic Role Labeling via Large Language Models

2025-06-03

AI Summary

Generative large language models (LLMs) have consistently underperformed encoder-decoder models (e.g., BERT-based architectures) on semantic role labeling (SRL), a core structured prediction task. Method: To bridge this gap, we propose a dual-mechanism framework: (1) retrieval-augmented generation guided by predicate-argument structure knowledge to incorporate external linguistic evidence, and (2) a structured self-correction module that iteratively validates and refines predictions based on output consistency. Contribution/Results: Our approach achieves the first comprehensive SOTA performance for LLMs on three major SRL benchmarks—CPB1.0 (Chinese), CoNLL-2009, and CoNLL-2012 (English)—surpassing all prior BERT-based models. It represents the first systematic breakthrough in overcoming LLM limitations for SRL and establishes the first generative paradigm demonstrably superior to conventional encoder-decoder architectures on this task.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Semantic role labeling (SRL) is a crucial task of natural language processing (NLP). Although generative decoder-based large language models (LLMs) have achieved remarkable success across various NLP tasks, they still lag behind state-of-the-art encoder-decoder (BERT-like) models in SRL. In this work, we seek to bridge this gap by equipping LLMs for SRL with two mechanisms: (a) retrieval-augmented generation and (b) self-correction. The first mechanism enables LLMs to leverage external linguistic knowledge such as predicate and argument structure descriptions, while the second allows LLMs to identify and correct inconsistent SRL outputs. We conduct extensive experiments on three widely-used benchmarks of SRL (CPB1.0, CoNLL-2009, and CoNLL-2012). Results demonstrate that our method achieves state-of-the-art performance in both Chinese and English, marking the first successful application of LLMs to surpass encoder-decoder approaches in SRL.

Problem

• Bridging performance gap between LLMs and encoder-decoder models in SRL
• Enhancing LLMs for SRL with retrieval-augmented generation
• Improving SRL accuracy via self-correction mechanisms in LLMs

Innovation

• Retrieval-augmented generation for external knowledge
• Self-correction mechanism for output consistency
• LLMs surpass encoder-decoder models in SRL