From Annotation to Adaptation: Metrics, Synthetic Data, and Aspect Extraction for Aspect-Based Sentiment Analysis with Large Language Models

2025-03-26

AI Summary

Implicit aspect extraction in Aspect-Based Sentiment Analysis (ABSA) remains challenging due to the scarcity of real-world annotated data, particularly for emerging domains. Method: This study investigates the capability boundaries of large language models (LLMs) on implicit aspect extraction in a novel domain—sports—and introduces the first synthetic, generation-oriented dataset tailored for generative LLMs. It reformulates ABSA as a structured generation task and proposes a novel aspect–polarity pair evaluation metric. Systematic zero-shot and few-shot evaluations are conducted on open-source LLMs (e.g., LLaMA, Phi). Contribution/Results: Results demonstrate that LLMs possess preliminary capacity for implicit aspect identification, yet exhibit limited cross-domain generalization. The proposed metric significantly enhances comparability and interpretability of generated outputs. This work establishes a new paradigm for data construction and evaluation in low-resource ABSA, bridging the gap between generative modeling and fine-grained sentiment analysis.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Generation
• Application Domains: Humanities & Computational Social Science

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems

Abstract

This study examines the performance of Large Language Models (LLMs) in Aspect-Based Sentiment Analysis (ABSA), with a focus on implicit aspect extraction in a novel domain. Using a synthetic sports feedback dataset, we evaluate open-weight LLMs' ability to extract aspect-polarity pairs and propose a metric to facilitate the evaluation of aspect extraction with generative models. Our findings highlight both the potential and limitations of LLMs in the ABSA task.

Problem

• Evaluating LLMs for aspect extraction in ABSA
• Proposing metrics for generative model evaluation
• Assessing LLMs' performance with synthetic sports data

Innovation

• Synthetic data for ABSA evaluation
• Metric for aspect extraction assessment
• LLMs for implicit aspect extraction