Problem
Research questions and friction points this paper is trying to address.
Limitations
NLP Research
Self-Reported Limitations
ACL
EMNLP
What Limits Us? Analyzing Self-Reported Limitations in NLP Research
🤖 AI Summary
本文通过一种新颖的人机协作框架，分析了2020至2025年间ACL和EMNLP会议论文中的局限性部分，探讨了自然语言处理研究中自我报告的限制及其趋势。
📝 Abstract
Since late 2022, a Limitations section has become mandatory at many top-tier NLP conferences. The growing number of accepted papers at these venues has resulted in a vast corpus of self-reported limitations that cannot all be manually reviewed, yet remains systematically unanalyzed. Therefore, in this paper, we conduct a large-scale analysis of the Limitations sections from ACL and EMNLP papers published between 2020 and 2025 to understand what researchers disclose about their own work. To do so, we implement a novel human-AI framework for iterative hybrid qualitative coding. This framework enables us to investigate trends in self-reported limitations over time, their correlations with specific paper attributes, and the writing patterns that recur around these disclosures. Our findings offer a critical reflection on the diverse reported challenges as well as the self-reporting practices of researchers in the NLP community.
Innovation
Methods, ideas, or system contributions that make the work stand out.
human-AI framework
iterative hybrid qualitative coding
self-reported limitations
🔎 Similar Papers
No similar papers found.
