Academic collaboration on large language model studies increases overall but varies across disciplines

2024-08-08

arXiv.org

AI Summary

This study investigates the impact of ChatGPT’s release on interdisciplinary scientific collaboration. Leveraging OpenAlex, we construct multilevel collaboration networks comprising 59,000 LLM-related papers and matched controls. We pioneer the integration of Shannon entropy—measuring institutional and departmental collaboration diversity—with a difference-in-differences (DID) framework to causally identify collaboration dynamics following the LLM shock. Results reveal a significant, gradient increase in collaboration entropy between computer science and social sciences; emerging synergy between medicine and CS; and the emergence of Stanford, Harvard, and UCL as critical network hubs. Methodologically, this work innovates by embedding information-theoretic entropy into scientometric collaboration analysis and bridging macro-level policy shocks with micro-level network structural inference. The findings provide robust empirical evidence for AI-driven transformation of scientific research paradigms.

Technology Category

• Application Domains: Humanities & Computational Social Science
• Machine Learning: Large Multimodal Models (LMMs)
• Cognitive Modeling & Cognitive Systems: Social Cognition And Interaction

Application Category

• Social Networks and Social Media: Computational social science
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Interdisciplinary collaboration is crucial for addressing complex scientific challenges. Recent advancements in large language models (LLMs) have shown significant potential in benefiting researchers across various fields. To explore their potential for interdisciplinary collaboration, we collect and analyze data from OpenAlex, an open-source academic database. Our dataset comprises 59,293 LLM-related papers, along with 70,945 machine learning (ML) papers and 73,110 papers from non-LLM/ML fields as control groups. We first employ Shannon Entropy to assess the diversity of collaboration. Our results reveal that many fields have exhibited a more significant increasing trend following the release of ChatGPT as compared to the control groups. In particular, Computer Science and Social Science display a consistent increase in both institution and department entropy. Other fields such as Decision Science, Psychology, and Health Professions have shown minor to significant increases. Our difference-in-difference analysis also indicates that the release of ChatGPT leads to a statistically significant increase in collaboration in several fields, such as Computer Science and Social Science. In addition, we analyze the author networks and find that Computer Science, Medicine, and other Computer Science-related departments are the most prominent. Regarding authors' institutions, our analysis reveals that entities such as Stanford University, Harvard University, and University College London are key players, either dominating centrality or playing crucial roles in connecting research networks. Overall, this study provides valuable information on the current landscape and evolving dynamics of collaboration networks in LLM research. It also suggests potential areas for fostering more diverse collaborations and highlights the need for continued research on the impact of LLMs on scientific practices.

Problem

• Large Language Models
• Interdisciplinary Collaboration
• Scientific Research

Innovation

• Large Language Models
• Interdisciplinary Research Collaboration
• Shannon Entropy and Difference-in-Differences Analysis