Affective Computing in the Era of Large Language Models: A Survey from the NLP Perspective

2024-07-30

arXiv.org

AI Summary

This study addresses critical challenges in leveraging large language models (LLMs) for affective computing (AC), including poor generalization, insufficient emotional diversity, and weak contextual adaptability. Methodologically, it establishes the first systematic, NLP-oriented framework for LLM-driven affective computing, proposing a unified taxonomy for affective understanding (AU) and affective generation (AG). It comprehensively surveys instruction tuning paradigms (full-parameter fine-tuning, LoRA, P-Tuning, Prompt Tuning), prompt engineering strategies (zero-/few-shot, chain-of-thought, agent-based prompting), and LLM architectural applications. Synthesizing over 100 state-of-the-art works, the study constructs a multi-scenario AC evaluation benchmark. Key findings reveal novel mechanistic insights—particularly LoRA’s parameter-efficient adaptation, chain-of-thought’s role in emotional reasoning, and agent-based prompting’s capacity for dynamic affective interaction—while rigorously characterizing LLMs’ advances and limitations in generalization, emotional expressiveness, and interpretability.

Technology Category

• Cognitive Modeling & Cognitive Systems: Affective Computing
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

Affective Computing (AC), integrating computer science, psychology, and cognitive science knowledge, aims to enable machines to recognize, interpret, and simulate human emotions.To create more value, AC can be applied to diverse scenarios, including social media, finance, healthcare, education, etc. Affective Computing (AC) includes two mainstream tasks, i.e., Affective Understanding (AU) and Affective Generation (AG). Fine-tuning Pre-trained Language Models (PLMs) for AU tasks has succeeded considerably. However, these models lack generalization ability, requiring specialized models for specific tasks. Additionally, traditional PLMs face challenges in AG, particularly in generating diverse and emotionally rich responses. The emergence of Large Language Models (LLMs), such as the ChatGPT series and LLaMA models, brings new opportunities and challenges, catalyzing a paradigm shift in AC. LLMs possess capabilities of in-context learning, common sense reasoning, and advanced sequence generation, which present unprecedented opportunities for AU. To provide a comprehensive overview of AC in the LLMs era from an NLP perspective, we summarize the development of LLMs research in this field, aiming to offer new insights. Specifically, we first summarize the traditional tasks related to AC and introduce the preliminary study based on LLMs. Subsequently, we outline the relevant techniques of popular LLMs to improve AC tasks, including Instruction Tuning and Prompt Engineering. For Instruction Tuning, we discuss full parameter fine-tuning and parameter-efficient methods such as LoRA, P-Tuning, and Prompt Tuning. In Prompt Engineering, we examine Zero-shot, Few-shot, Chain of Thought (CoT), and Agent-based methods for AU and AG. To clearly understand the performance of LLMs on different Affective Computing tasks, we further summarize the existing benchmarks and evaluation methods.

Problem

• Improving affective understanding and generation across diverse tasks and domains
• Enhancing emotional response diversity and appropriateness in language models
• Addressing generalization limitations in emotion-aware AI systems

Innovation

• Instruction Tuning and LoRA for parameter-efficient adaptation
• Prompt Engineering with chain-of-thought and agent-based methods
• Reinforcement Learning from human and AI feedback for optimization