Sarc7: Evaluating Sarcasm Detection and Generation with Seven Types and Emotion-Informed Techniques

2025-05-31

AI Summary

Fine-grained modeling of sarcasm remains challenging due to semantic incongruity, affective intensity, and strong contextual dependency—particularly in recognition and generation across diverse sarcastic subtypes. Method: We introduce Sarc7, the first benchmark dataset covering seven sarcasm categories (self-deprecating, reflective, deadpan, polite, disgusted, angry, manic) and propose an affect-aware prompting paradigm that explicitly integrates this seven-dimensional sarcasm typology with explicit emotional cues into zero-shot and few-shot recognition and generation. Our approach combines affect-enhanced prompt engineering, incongruity modeling, and context-aware generation strategies. Results: On Sarc7, Gemini 2.5 achieves an F1 score of 0.3664 under emotion-aware prompting—the highest among all settings. Human evaluation shows a 38.46% improvement in sarcastic utterance generation success over zero-shot baselines. This work establishes an interpretable, scalable framework for fine-grained sarcasm understanding and generation.

Technology Category

• Natural Language Processing: Generation
• Cognitive Modeling & Cognitive Systems: Affective Computing
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Sarcasm is a form of humor where expressions convey meanings opposite to their literal interpretations. Classifying and generating sarcasm using large language models is vital for interpreting human communication. Sarcasm poses challenges for computational models, due to its nuanced nature. We introduce Sarc7, a benchmark that classifies 7 types of sarcasm: self-deprecating, brooding, deadpan, polite, obnoxious, raging, and manic by annotating entries of the MUStARD dataset. Classification was evaluated using zero-shot, few-shot, chain-of-thought (CoT), and a novel emotion-based prompting technique. We propose an emotion-based generation method developed by identifying key components of sarcasm-incongruity, shock value, and context dependency. Our classification experiments show that Gemini 2.5, using emotion-based prompting, outperforms other setups with an F1 score of 0.3664. Human evaluators preferred our emotion-based prompting, with 38.46% more successful generations than zero-shot prompting.

Problem

• Classifying seven types of sarcasm in computational models
• Improving sarcasm detection using emotion-informed techniques
• Generating sarcasm with incongruity and context dependency

Innovation

• Classifies 7 sarcasm types using annotated MUStARD dataset
• Uses emotion-based prompting for improved classification
• Generates sarcasm via incongruity, shock value, context dependency