ArgInstruct: Specialized Instruction Fine-Tuning for Computational Argumentation

2025-05-28

AI Summary

Large language models (LLMs) exhibit limited generalization and fragile cross-task transferability in computational argumentation (CA). Method: We propose the first systematic instruction-tuning framework tailored to CA, comprising: (1) synthesizing 52K fine-grained instructions covering 105 distinct CA tasks via Self-Instruct; (2) constructing the first multi-task, formalism-driven CA-specific instruction benchmark; and (3) designing task-aware prompting templates and domain-adaptive fine-tuning strategies. Contribution/Results: Our approach achieves, for the first time, simultaneous preservation of both CA-domain specialization and broad generalization—demonstrated by strong performance on the SuperNI benchmark. It significantly improves model accuracy across both in-distribution and zero-shot CA tasks, empirically validating that domain-specific enhancement need not compromise generalization capability.

Technology Category

• Knowledge Representation and Reasoning: Argumentation
• Machine Learning: Large Multimodal Models (LMMs)
• Cognitive Modeling & Cognitive Systems: Computational Creativity

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Large language models for search
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Training large language models (LLMs) to follow instructions has significantly enhanced their ability to tackle unseen tasks. However, despite their strong generalization capabilities, instruction-following LLMs encounter difficulties when dealing with tasks that require domain knowledge. This work introduces a specialized instruction fine-tuning for the domain of computational argumentation (CA). The goal is to enable an LLM to effectively tackle any unseen CA tasks while preserving its generalization capabilities. Reviewing existing CA research, we crafted natural language instructions for 105 CA tasks to this end. On this basis, we developed a CA-specific benchmark for LLMs that allows for a comprehensive evaluation of LLMs' capabilities in solving various CA tasks. We synthesized 52k CA-related instructions, adapting the self-instruct process to train a CA-specialized instruction-following LLM. Our experiments suggest that CA-specialized instruction fine-tuning significantly enhances the LLM on both seen and unseen CA tasks. At the same time, performance on the general NLP tasks of the SuperNI benchmark remains stable.

Problem

• Enhancing LLMs for computational argumentation tasks
• Creating specialized instruction dataset for CA tasks
• Improving CA performance without losing general NLP capabilities

Innovation

• Specialized instruction fine-tuning for argumentation
• Synthesized 52k CA-related instructions dataset
• CA-specific benchmark for comprehensive LLM evaluation