Has ACL Lost Its Crown? A Decade-Long Quantitative Analysis of Scale and Impact Across Leading AI Conferences

2025-12-03

AI Summary

Rapid expansion of AI/NLP conferences has raised concerns about declining review quality and diluted scholarly impact. Method: We conduct a decade-long (2014–2023) empirical analysis across seven top-tier conferences, introducing the novel metric “quality–quantity elasticity” (citation growth per acceptance growth) and establishing a four-dimensional bibliometric framework—encompassing citation intensity, impact distribution, cross-conference diffusion, and temporal decay—to enable longitudinal, cross-conference, data-driven comparison. Contribution/Results: Our analysis reveals pronounced impact stratification and divergent expansion efficiency among NLP venues; ML conferences maintain robust dominance; while AI conferences exhibit structural decline. The study delivers quantifiable empirical benchmarks and methodological tools for academic conference governance and evaluation.

Technology Category

• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Machine Learning: Evaluation and Analysis

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Web Mining and Content Analysis: Web data quality in the era of algorithmically-generated content
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

The recent surge of language models has rapidly expanded NLP research, driving an exponential rise in submissions and acceptances at major conferences. Yet this growth has been shadowed by escalating concerns over conference quality, e.g., plagiarism, reviewer inexperience and collusive bidding. However, existing studies rely largely on qualitative accounts (e.g., expert interviews, social media discussions, etc.), lacking longitudinal empirical evidence. To fill this gap, we conduct a ten year empirical study spanning seven leading conferences. We build a four dimensional bibliometric framework covering conference scale, core citation statistics,impact dispersion, cross venue and journal influence, etc. Notably, we further propose a metric Quality Quantity Elasticity, which measures the elasticity of citation growth relative to acceptance growth. Our findings show that ML venues sustain dominant and stable impact, NLP venues undergo widening stratification with mixed expansion efficiency, and AI venues exhibit structural decline. This study provides the first decade-long, cross-venue empirical evidence on the evolution of major conferences.

Problem

• Analyzes how major AI conferences evolved in scale and impact over a decade
• Measures citation growth elasticity relative to acceptance growth across venues
• Compares impact stratification and expansion efficiency across seven leading conferences

Innovation

• Four-dimensional bibliometric framework for conference analysis
• Proposed Quality Quantity Elasticity metric for citation growth
• Ten-year empirical study across seven leading AI conferences