How broad is that claim? Mapping Generalisation in NLP Research

2026-09-13

AI Summary

研究提出NLPGenX分类法和基于LLM的框架NLPGenA，自动识别并分类科学文章中的泛化声明，以帮助检测过度泛化和可能的科研结果误述。

Abstract

Generalisations are common in scientific communication, even though they are semantically ambiguous. An automated method is needed to identify and categorise claims according to their level of generalisation, in order help detect an over-reliance on generalisations and possible misrepresentations of scientific findings. We introduce a comprehensive taxonomy of generalisations in the scientific domain, NLPGenX, which labels claims according to their level of generality and framing within the text. We operationalise this taxonomy with an LLM-powered framework, NLPGenA, that automatically classifies sentences from scientific articles into 5 different generalisation classes. We validate our framework with human annotators and use the framework to construct a large-scale dataset of NLP papers annotated according to generality, with auxiliary labels for hedging and vague descriptors (NLPGens). We use NLPGens to analyse the use of generalisations in NLP papers across multiple venues and subdomains, and to examine associations with citation counts, hedging, and vague descriptors.

Problem

• Generalisations
• Scientific Communication
• Semantic Ambiguity
• Automation
• Classification

Innovation

• NLPGenX
• NLPGenA
• generalisation classification
• automated framework
• NLPGens