Large Language Models Meet NLP: A Survey

2024-05-21

arXiv.org

AI Summary

This study systematically investigates the current state and evolutionary trajectory of large language models (LLMs) in natural language processing (NLP), addressing three core questions: (1) How are LLMs applied to NLP tasks? (2) Have classical NLP tasks been fundamentally resolved? (3) What are the future research directions? To this end, we propose the first unified classification framework for LLM applications in NLP, dichotomizing methodologies into parameter-freezing and parameter-finetuning paradigms. We conduct a comprehensive empirical assessment of LLMs’ coverage and performance bottlenecks across canonical NLP tasks. Furthermore, we synthesize emerging frontiers—including reasoning augmentation and multimodal integration—as well as persistent challenges such as interpretability and long-range dependency modeling. Grounded in a rigorous literature review and technical evolution analysis, we construct the first structured, holistic research landscape map of LLM-NLP, offering both theoretical insight and practical guidance for the community.

Technology Category

• Natural Language Processing: (Large) Language Models
• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Large Vision Models

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Large language models for search

Abstract

While large language models (LLMs) like ChatGPT have shown impressive capabilities in Natural Language Processing (NLP) tasks, a systematic investigation of their potential in this field remains largely unexplored. This study aims to address this gap by exploring the following questions: (1) How are LLMs currently applied to NLP tasks in the literature? (2) Have traditional NLP tasks already been solved with LLMs? (3) What is the future of the LLMs for NLP? To answer these questions, we take the first step to provide a comprehensive overview of LLMs in NLP. Specifically, we first introduce a unified taxonomy including (1) parameter-frozen application and (2) parameter-tuning application to offer a unified perspective for understanding the current progress of LLMs in NLP. Furthermore, we summarize the new frontiers and the associated challenges, aiming to inspire further groundbreaking advancements. We hope this work offers valuable insights into the {potential and limitations} of LLMs in NLP, while also serving as a practical guide for building effective LLMs in NLP.

Problem

• Systematic investigation of LLMs' potential in NLP tasks
• Current application status of LLMs in NLP literature
• Future directions and challenges for LLMs in NLP

Innovation

• Unified taxonomy for LLM paradigms
• Parameter-frozen and tuning paradigms
• Comprehensive survey of LLM applications