R1-RE: Cross-Domain Relationship Extraction with RLVR

2025-07-06

AI Summary

Relation extraction (RE) exhibits poor generalization in out-of-domain (OOD) settings, as conventional supervised methods suffer from domain bias. To address this, we propose R1-RE—a novel framework that reformulates RE as a multi-step reasoning task guided by human-written annotation guidelines. R1-RE introduces verifiable-reward reinforcement learning (RLVR), the first RL paradigm explicitly designed to emulate human annotation logic by dynamically validating and optimizing intermediate reasoning steps. Crucially, it leverages compact language models (e.g., 7B-parameter variants) augmented with external annotation guidelines as structured prior knowledge, thereby enhancing cross-domain robustness without scaling model size. Evaluated on SemEval-2010 and MDKG, R1-RE-7B achieves an average cross-domain accuracy of 70%, matching the performance of large proprietary models such as GPT-4o. This demonstrates that controllable, guideline-driven reasoning enables lightweight models to attain strong OOD generalization—establishing a new paradigm for efficient and interpretable RE.

Technology Category

• Knowledge Representation and Reasoning: Argumentation
• Natural Language Processing: Safety and Robustness
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling

Abstract

Relationship extraction (RE) is a core task in natural language processing. Traditional approaches typically frame RE as a supervised learning problem, directly mapping context to labels-an approach that often suffers from poor out-of-domain (OOD) generalization. Inspired by the workflow of human annotators, we reframe RE as a reasoning task guided by annotation guidelines and introduce R1-RE, the first reinforcement learning with verifiable reward (RLVR) framework for RE tasks. Our method elicits the reasoning abilities of small language models for annotation tasks, resulting in significantly improved OOD robustness. We evaluate our approach on the public Sem-2010 dataset and a private MDKG dataset. The R1-RE-7B model attains an average OOD accuracy of approximately 70%, on par with leading proprietary models such as GPT-4o. Additionally, our comprehensive analysis provides novel insights into the training dynamics and emergent reasoning behaviors of the RLVR paradigm for RE.

Problem

• Improves out-of-domain generalization in relationship extraction
• Reframes RE as reasoning task with verifiable reinforcement learning
• Enhances small models' reasoning for annotation tasks

Innovation

• Reinforcement learning with verifiable reward framework
• Reframing RE as reasoning task with guidelines
• Improved OOD robustness via small language models